Jul 22 02:19:18.093469 ip-10-0-142-149 systemd[1]: Starting Kubernetes Kubelet... Jul 22 02:19:18.530296 ip-10-0-142-149 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:18.530296 ip-10-0-142-149 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 22 02:19:18.530296 ip-10-0-142-149 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:18.530296 ip-10-0-142-149 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 22 02:19:18.530296 ip-10-0-142-149 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 22 02:19:18.531002 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.530353 2570 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539798 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539820 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539824 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539828 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539831 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539834 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.539827 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539837 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539840 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539843 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539846 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539849 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539851 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539854 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539857 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539859 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539862 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539865 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539867 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539870 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539873 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539875 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539878 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539880 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539883 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539885 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.540073 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539887 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539890 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539892 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539895 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539897 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539900 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539909 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539912 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539914 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539917 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539919 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539922 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539924 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539926 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539929 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539932 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539935 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539937 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539940 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539942 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.540561 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539945 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539947 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539950 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539953 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539956 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539960 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539963 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539965 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539968 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539970 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539972 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539975 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539977 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539981 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539986 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539989 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539993 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539995 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.539998 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540001 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.541046 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540004 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540006 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540009 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540011 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540014 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540016 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540019 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540021 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540025 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540029 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540031 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540034 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540037 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540039 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540043 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540046 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540049 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540051 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540055 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.541547 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540057 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540060 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540062 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540064 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540068 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540071 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540074 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540076 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540079 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540082 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540084 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540087 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540089 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540091 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540094 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540804 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540812 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540815 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540818 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540821 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.542015 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540824 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540827 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540830 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540832 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540835 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540838 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540840 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540843 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540845 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540848 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540850 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540852 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540856 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540858 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540860 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540863 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540866 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540868 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540871 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540873 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.542862 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540876 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540878 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540880 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540883 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540885 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540888 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540890 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540893 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540896 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540899 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540902 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540905 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540908 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540910 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540913 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540915 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540918 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540920 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540923 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540925 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.543387 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540928 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540930 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540933 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540935 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540937 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540941 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540943 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540946 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540949 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540951 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540954 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540956 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540958 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540961 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540963 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540965 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540969 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540971 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540974 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.543869 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540976 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540979 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540981 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540984 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540986 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540989 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540991 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540994 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540997 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.540999 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541002 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541004 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541006 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541009 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541011 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541014 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541016 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541019 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541021 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.544347 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541025 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541028 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541031 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541034 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541036 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541039 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541041 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541044 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541047 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541049 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541052 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541056 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541060 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541063 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541066 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541069 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541460 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541466 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541469 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541472 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.544818 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541475 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541478 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541481 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541484 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541486 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541489 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541491 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541494 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541497 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541499 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541502 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541506 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541509 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541512 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541515 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541518 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541521 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541523 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541527 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.545315 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541529 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541532 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541534 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541536 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541539 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541542 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541544 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541547 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541550 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541552 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541556 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541558 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541561 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541564 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541566 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541569 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541573 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541576 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541578 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.545782 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541580 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541583 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541585 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541587 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541590 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541592 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541594 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541603 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541606 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541609 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541611 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541614 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541616 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541619 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541622 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541626 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541629 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541632 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541635 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.546265 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541637 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541640 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541643 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541645 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541648 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541650 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541653 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541655 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541657 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541660 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541662 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541665 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541667 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541670 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541672 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541674 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541677 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541679 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541682 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541684 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.546721 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541687 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541690 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541693 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541695 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541698 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541700 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541703 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541705 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541708 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541711 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541714 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541716 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541719 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541721 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541724 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541726 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541729 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.541734 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541807 2570 flags.go:64] FLAG: --address="0.0.0.0" Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541814 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541820 2570 flags.go:64] FLAG: --anonymous-auth="true" Jul 22 02:19:18.547250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541824 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541829 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541832 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541836 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541840 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541843 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541845 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541849 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541852 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541855 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541857 2570 flags.go:64] FLAG: --cgroup-root="" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541860 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541863 2570 flags.go:64] FLAG: --client-ca-file="" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541867 2570 flags.go:64] FLAG: --cloud-provider="external" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541870 2570 flags.go:64] FLAG: --cluster-dns="[]" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541873 2570 flags.go:64] FLAG: --cluster-domain="" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541876 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541879 2570 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541881 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541885 2570 flags.go:64] FLAG: --container-log-max-files="5" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541888 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541891 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541893 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 22 02:19:18.547747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541896 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541899 2570 flags.go:64] FLAG: --contention-profiling="false" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541902 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541905 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541908 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541913 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541917 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541920 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541923 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541925 2570 flags.go:64] FLAG: --enable-load-reader="false" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541928 2570 flags.go:64] FLAG: --enable-server="true" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541931 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541940 2570 flags.go:64] FLAG: --event-burst="100" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541943 2570 flags.go:64] FLAG: --event-qps="50" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541946 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541949 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541952 2570 flags.go:64] FLAG: --eviction-hard="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541956 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541958 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541961 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541964 2570 flags.go:64] FLAG: --eviction-soft="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541967 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541969 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541982 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541985 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jul 22 02:19:18.548308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541988 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541991 2570 flags.go:64] FLAG: --fail-swap-on="true" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541994 2570 flags.go:64] FLAG: --feature-gates="" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.541997 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542000 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542003 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542006 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542009 2570 flags.go:64] FLAG: --healthz-port="10248" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542012 2570 flags.go:64] FLAG: --help="false" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542015 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542018 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542021 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542027 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542030 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542034 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542037 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542040 2570 flags.go:64] FLAG: --image-service-endpoint="" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542042 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542045 2570 flags.go:64] FLAG: --kube-api-burst="100" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542048 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542051 2570 flags.go:64] FLAG: --kube-api-qps="50" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542054 2570 flags.go:64] FLAG: --kube-reserved="" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542057 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542059 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 22 02:19:18.548943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542062 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542065 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542067 2570 flags.go:64] FLAG: --lock-file="" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542070 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542073 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542075 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542081 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542084 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542087 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542089 2570 flags.go:64] FLAG: --logging-format="text" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542092 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542095 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542098 2570 flags.go:64] FLAG: --manifest-url="" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542101 2570 flags.go:64] FLAG: --manifest-url-header="" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542105 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542108 2570 flags.go:64] FLAG: --max-open-files="1000000" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542112 2570 flags.go:64] FLAG: --max-pods="110" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542116 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542119 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542122 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542126 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542223 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542227 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542230 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542238 2570 flags.go:64] FLAG: --node-status-max-images="50" Jul 22 02:19:18.549643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542241 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542244 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542247 2570 flags.go:64] FLAG: --pod-cidr="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542250 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542255 2570 flags.go:64] FLAG: --pod-manifest-path="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542258 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542260 2570 flags.go:64] FLAG: --pods-per-core="0" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542263 2570 flags.go:64] FLAG: --port="10250" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542266 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542269 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00251ebd19f6b882e" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542273 2570 flags.go:64] FLAG: --qos-reserved="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542275 2570 flags.go:64] FLAG: --read-only-port="10255" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542278 2570 flags.go:64] FLAG: --register-node="true" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542281 2570 flags.go:64] FLAG: --register-with-taints="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542284 2570 flags.go:64] FLAG: --registry-burst="10" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542288 2570 flags.go:64] FLAG: --registry-qps="5" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542291 2570 flags.go:64] FLAG: --reserved-cpus="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542294 2570 flags.go:64] FLAG: --reserved-memory="" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542297 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542300 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542303 2570 flags.go:64] FLAG: --rotate-certificates="false" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542305 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542308 2570 flags.go:64] FLAG: --runonce="false" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542311 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542314 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 22 02:19:18.550817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542318 2570 flags.go:64] FLAG: --seccomp-default="false" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542321 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542325 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542328 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542331 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542333 2570 flags.go:64] FLAG: --storage-driver-password="root" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542336 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542339 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542341 2570 flags.go:64] FLAG: --storage-driver-user="root" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542344 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542347 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542350 2570 flags.go:64] FLAG: --system-cgroups="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542352 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542358 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542360 2570 flags.go:64] FLAG: --tls-cert-file="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542363 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542367 2570 flags.go:64] FLAG: --tls-min-version="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542369 2570 flags.go:64] FLAG: --tls-private-key-file="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542372 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542375 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542378 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542381 2570 flags.go:64] FLAG: --v="2" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542389 2570 flags.go:64] FLAG: --version="false" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542394 2570 flags.go:64] FLAG: --vmodule="" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542397 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 22 02:19:18.551856 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.542401 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542514 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542517 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542520 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542523 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542526 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542529 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542535 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542538 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542542 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542544 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542547 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542549 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542552 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542554 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542557 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542560 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542562 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542565 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542567 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.552506 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542570 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542573 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542575 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542578 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542580 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542582 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542585 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542587 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542590 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542592 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542595 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542597 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542600 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542602 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542605 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542607 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542610 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542612 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542615 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.553426 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542618 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542621 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542625 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542627 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542630 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542632 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542636 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542639 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542641 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542644 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542646 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542649 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542651 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542653 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542656 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542658 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542661 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542663 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542665 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.554253 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542668 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542671 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542673 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542676 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542678 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542681 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542683 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542686 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542688 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542690 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542693 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542695 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542698 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542701 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542704 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542707 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542710 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542712 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542715 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542717 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.554733 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542720 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542722 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542725 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542728 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542730 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542732 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542735 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542737 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542740 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542742 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542745 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542747 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542750 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542752 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542754 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542759 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542763 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542767 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542769 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.555427 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542772 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542775 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.542778 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.543452 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.551759 2570 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.551782 2570 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.552695 2570 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.552709 2570 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552800 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552807 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552812 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552816 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552821 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552825 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552829 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.556223 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552833 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552838 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552842 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552846 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552851 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552855 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552859 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552863 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552867 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552871 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552875 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552879 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552883 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552887 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552892 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552896 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552900 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552904 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552908 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552912 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.556699 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552916 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552920 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552927 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552931 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552935 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552940 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552943 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552947 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552951 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552955 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552959 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552963 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552968 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552971 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552976 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552982 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552987 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552992 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.552996 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.557224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553000 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553005 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553009 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553013 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553017 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553021 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553026 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553030 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553034 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553038 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553042 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553047 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553052 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553056 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553061 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553065 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553071 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553076 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553080 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553084 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.557789 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553091 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553097 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553102 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553106 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553110 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553114 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553120 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553126 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553149 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553154 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553158 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553162 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553167 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553171 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553174 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553178 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553182 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553186 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553190 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553194 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.558318 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553197 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553200 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553204 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553208 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553212 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553216 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553220 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553224 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553228 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553233 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553236 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553240 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553244 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.553251 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:18.558796 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553438 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553445 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553450 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553454 2570 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553458 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553462 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553467 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553471 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553477 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553483 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553488 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553492 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553497 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553501 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553506 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553510 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553515 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553519 2570 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553523 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 22 02:19:18.559201 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553528 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553532 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553537 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553541 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553546 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553551 2570 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553555 2570 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553559 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553563 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553568 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553572 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553576 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553580 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553584 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553588 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553593 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553597 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553601 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553605 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553610 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 22 02:19:18.559659 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553613 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553618 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553622 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553625 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553629 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553633 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553638 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553642 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553646 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553650 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553655 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553659 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553663 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553668 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553671 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553675 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553682 2570 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553687 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553692 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 22 02:19:18.560182 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553696 2570 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553701 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553705 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553710 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553714 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553718 2570 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553723 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553726 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553731 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553735 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553739 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553743 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553747 2570 feature_gate.go:328] unrecognized feature gate: Example Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553751 2570 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553755 2570 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553759 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553763 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553767 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553771 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553775 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553779 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 22 02:19:18.560641 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553783 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553787 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553792 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553796 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553800 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553804 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553808 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553812 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553816 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553822 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553828 2570 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553833 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553837 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553841 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553845 2570 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553850 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553854 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553858 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553862 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 22 02:19:18.561224 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:18.553866 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.553873 2570 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.554056 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.557259 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.559118 2570 server.go:1019] "Starting client certificate rotation" Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.559237 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:19:18.561674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.559271 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 22 02:19:18.563080 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.563062 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:19:18.570151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.570117 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 22 02:19:18.581113 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.581089 2570 log.go:25] "Validated CRI v1 runtime API" Jul 22 02:19:18.584948 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.584933 2570 log.go:25] "Validated CRI v1 image API" Jul 22 02:19:18.585877 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.585850 2570 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 22 02:19:18.590790 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.590767 2570 fs.go:135] Filesystem UUIDs: map[77740305-fbb8-4aec-b445-d7799417778e:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 ecfb8290-2a7c-4bc6-bfea-6f25eff7ef66:/dev/nvme0n1p4] Jul 22 02:19:18.590848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.590791 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 22 02:19:18.595823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.595713 2570 manager.go:216] Machine: {Timestamp:2026-07-22 02:19:18.594392855 +0000 UTC m=+0.372468786 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3094759 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec23fde5f822b0c717a5faf0ed0cede4 SystemUUID:ec23fde5-f822-b0c7-17a5-faf0ed0cede4 BootID:70f3fdd2-bf2f-4b65-8f6b-be48b93ec6f0 Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:86:e8:e5:26:03 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:86:e8:e5:26:03 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:06:a2:ef:3c:16:7f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 22 02:19:18.595823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.595814 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 22 02:19:18.595979 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.595888 2570 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 22 02:19:18.597747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.597728 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:19:18.599305 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.599274 2570 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 22 02:19:18.599468 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.599311 2570 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-149.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 22 02:19:18.600188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.600176 2570 topology_manager.go:138] "Creating topology manager with none policy" Jul 22 02:19:18.600241 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.600194 2570 container_manager_linux.go:309] "Creating device plugin manager" Jul 22 02:19:18.600241 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.600207 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:19:18.600241 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.600229 2570 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 22 02:19:18.602184 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.602171 2570 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 22 02:19:18.606879 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.606855 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:19:18.607027 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607015 2570 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 22 02:19:18.607083 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607054 2570 kubelet.go:486] "Attempting to sync node with API server" Jul 22 02:19:18.607083 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607068 2570 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 22 02:19:18.607185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607084 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 22 02:19:18.607185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607098 2570 kubelet.go:398] "Adding apiserver pod source" Jul 22 02:19:18.607185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607111 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 22 02:19:18.607496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607483 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:19:18.607555 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.607505 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 22 02:19:18.611719 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.611701 2570 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 22 02:19:18.613530 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.613517 2570 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 22 02:19:18.613577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.613548 2570 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 22 02:19:18.614314 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614302 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614321 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614327 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614333 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614340 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614345 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 22 02:19:18.614360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614355 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 22 02:19:18.614514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614363 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 22 02:19:18.614514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614372 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 22 02:19:18.614514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614377 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 22 02:19:18.614514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614385 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 22 02:19:18.614514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.614393 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 22 02:19:18.615278 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.615266 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 22 02:19:18.615323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.615281 2570 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 22 02:19:18.617795 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.617768 2570 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-149.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 22 02:19:18.618491 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.618469 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 22 02:19:18.618543 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.618468 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 22 02:19:18.618593 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.618545 2570 server.go:1268] "Started kubelet" Jul 22 02:19:18.618649 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.618603 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 22 02:19:18.618704 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.618625 2570 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 22 02:19:18.618704 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.618699 2570 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 22 02:19:18.619370 ip-10-0-142-149 systemd[1]: Started Kubernetes Kubelet. Jul 22 02:19:18.619975 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.619795 2570 server.go:310] "Adding debug handlers to kubelet server" Jul 22 02:19:18.620116 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.620098 2570 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 22 02:19:18.625956 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.624839 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-149.ec2.internal.18c47b26baccd917 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-149.ec2.internal,UID:ip-10-0-142-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-149.ec2.internal,},FirstTimestamp:2026-07-22 02:19:18.618519831 +0000 UTC m=+0.396595764,LastTimestamp:2026-07-22 02:19:18.618519831 +0000 UTC m=+0.396595764,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-149.ec2.internal,}" Jul 22 02:19:18.626403 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.626386 2570 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 22 02:19:18.631628 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.631614 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 22 02:19:18.631694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.631629 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 22 02:19:18.631755 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.631710 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:18.632459 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632397 2570 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 22 02:19:18.632459 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632431 2570 factory.go:55] Registering systemd factory Jul 22 02:19:18.632459 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632450 2570 factory.go:223] Registration of the systemd container factory successfully Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632434 2570 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632396 2570 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632634 2570 reconstruct.go:97] "Volume reconstruction finished" Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632642 2570 reconciler.go:29] "Reconciler: start to sync state" Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632656 2570 factory.go:153] Registering CRI-O factory Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632670 2570 factory.go:223] Registration of the crio container factory successfully Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632736 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632761 2570 factory.go:103] Registering Raw factory Jul 22 02:19:18.632815 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.632776 2570 manager.go:1196] Started watching for new ooms in manager Jul 22 02:19:18.633350 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.633238 2570 manager.go:349] Starting recovery of all containers Jul 22 02:19:18.634002 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.633958 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-149.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 22 02:19:18.634084 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.634063 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 22 02:19:18.641659 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.641625 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 22 02:19:18.645536 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.645516 2570 manager.go:354] Recovery completed Jul 22 02:19:18.646070 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.646047 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47x74" Jul 22 02:19:18.650111 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.650098 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.652262 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652249 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.652318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652277 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.652318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652288 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.652826 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652813 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 22 02:19:18.652826 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652824 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 22 02:19:18.652913 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652839 2570 state_mem.go:36] "Initialized new in-memory state store" Jul 22 02:19:18.652913 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.652877 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.653489 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.653475 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.653542 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.653503 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.653542 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.653515 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.653954 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.653939 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-47x74" Jul 22 02:19:18.656043 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.656028 2570 policy_none.go:49] "None policy: Start" Jul 22 02:19:18.656078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.656050 2570 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 22 02:19:18.656078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.656064 2570 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 22 02:19:18.657981 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.657960 2570 policy_none.go:47] "Start" Jul 22 02:19:18.661881 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.661708 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-149.ec2.internal.18c47b26bccfba59 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-149.ec2.internal,UID:ip-10-0-142-149.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-142-149.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-142-149.ec2.internal,},FirstTimestamp:2026-07-22 02:19:18.652263001 +0000 UTC m=+0.430338933,LastTimestamp:2026-07-22 02:19:18.652263001 +0000 UTC m=+0.430338933,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-149.ec2.internal,}" Jul 22 02:19:18.700807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.700780 2570 manager.go:337] "Starting Device Plugin manager" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.700814 2570 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.700824 2570 server.go:87] "Starting device plugin registration server" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.701073 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.701085 2570 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.701168 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.701242 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.701250 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.701794 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 22 02:19:18.722486 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.701837 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:18.742205 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.742180 2570 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 22 02:19:18.742317 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.742221 2570 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 22 02:19:18.742317 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.742247 2570 kubelet.go:2448] "Starting kubelet main sync loop" Jul 22 02:19:18.742317 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.742281 2570 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 22 02:19:18.744984 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.744962 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:18.802229 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.802112 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.803039 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.803023 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.803110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.803054 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.803110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.803070 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.803110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.803102 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.812971 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.812955 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.813068 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.812980 2570 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-149.ec2.internal\": node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:18.843199 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.843173 2570 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal"] Jul 22 02:19:18.843295 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.843242 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.843999 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.843985 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.844051 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.844012 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.844051 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.844027 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.846142 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.846117 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.846267 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.846252 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.846307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.846289 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.847052 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847034 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.847206 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847187 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.847362 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.847235 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:18.847470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847393 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.847583 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847479 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.847932 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847640 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.847932 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.847681 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.849871 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.849851 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.849960 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.849877 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:18.850572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.850557 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:18.850653 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.850588 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:18.850653 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.850601 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:18.872688 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.872666 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-149.ec2.internal\" not found" node="ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.876851 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.876835 2570 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-149.ec2.internal\" not found" node="ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.934681 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.934656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.934802 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.934684 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.934802 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:18.934706 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7e85b7071d89f6d9d1647bbf823b423-config\") pod \"kube-apiserver-proxy-ip-10-0-142-149.ec2.internal\" (UID: \"e7e85b7071d89f6d9d1647bbf823b423\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:18.948462 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:18.948441 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.035812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035771 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.035812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035815 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.036017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035834 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7e85b7071d89f6d9d1647bbf823b423-config\") pod \"kube-apiserver-proxy-ip-10-0-142-149.ec2.internal\" (UID: \"e7e85b7071d89f6d9d1647bbf823b423\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.036017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035873 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/e7e85b7071d89f6d9d1647bbf823b423-config\") pod \"kube-apiserver-proxy-ip-10-0-142-149.ec2.internal\" (UID: \"e7e85b7071d89f6d9d1647bbf823b423\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.036017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.036017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.035880 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/050ebd4d45e7f09a481ee99742944a20-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal\" (UID: \"050ebd4d45e7f09a481ee99742944a20\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.048915 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.048884 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.149811 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.149736 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.174935 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.174907 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.182066 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.182048 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:19.250502 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.250460 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.350961 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.350932 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.451602 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.451523 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.552060 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.552027 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.558193 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.558173 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 22 02:19:19.558321 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.558305 2570 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 22 02:19:19.637768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.637739 2570 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:19.652946 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.652921 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.661093 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.661062 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-21 02:14:19 +0000 UTC" deadline="2028-01-24 19:08:15.516245164 +0000 UTC" Jul 22 02:19:19.661093 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.661088 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13240h48m55.855160583s" Jul 22 02:19:19.728236 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:19.728199 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod050ebd4d45e7f09a481ee99742944a20.slice/crio-a252adb32c55d1a9901b956affd07158ed22fbae31d3a1715e12749356d26fec WatchSource:0}: Error finding container a252adb32c55d1a9901b956affd07158ed22fbae31d3a1715e12749356d26fec: Status 404 returned error can't find the container with id a252adb32c55d1a9901b956affd07158ed22fbae31d3a1715e12749356d26fec Jul 22 02:19:19.728439 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:19.728421 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7e85b7071d89f6d9d1647bbf823b423.slice/crio-592bcab96ab15ae9a394c6e8df7d3923c39140745362696d95ffba48b08b721c WatchSource:0}: Error finding container 592bcab96ab15ae9a394c6e8df7d3923c39140745362696d95ffba48b08b721c: Status 404 returned error can't find the container with id 592bcab96ab15ae9a394c6e8df7d3923c39140745362696d95ffba48b08b721c Jul 22 02:19:19.733412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.733394 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:19:19.745104 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.745060 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" event={"ID":"050ebd4d45e7f09a481ee99742944a20","Type":"ContainerStarted","Data":"a252adb32c55d1a9901b956affd07158ed22fbae31d3a1715e12749356d26fec"} Jul 22 02:19:19.745921 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.745901 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" event={"ID":"e7e85b7071d89f6d9d1647bbf823b423","Type":"ContainerStarted","Data":"592bcab96ab15ae9a394c6e8df7d3923c39140745362696d95ffba48b08b721c"} Jul 22 02:19:19.753023 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.752998 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.843573 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.843551 2570 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 22 02:19:19.844723 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.844710 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientMemory" Jul 22 02:19:19.844805 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.844736 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasNoDiskPressure" Jul 22 02:19:19.844805 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:19.844745 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeHasSufficientPID" Jul 22 02:19:19.853574 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.853556 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:19.954224 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:19.954174 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:20.054822 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.054753 2570 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-149.ec2.internal\" not found" Jul 22 02:19:20.080758 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.080732 2570 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:20.132095 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.132072 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" Jul 22 02:19:20.141992 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.141973 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:19:20.142907 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.142895 2570 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" Jul 22 02:19:20.154144 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.154120 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 22 02:19:20.160732 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.160716 2570 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:20.476195 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.476156 2570 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 22 02:19:20.608448 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.608362 2570 apiserver.go:52] "Watching apiserver" Jul 22 02:19:20.619748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.619720 2570 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 22 02:19:20.620128 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.620105 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-f6xrz","kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal","openshift-cluster-node-tuning-operator/tuned-7fcfs","openshift-image-registry/node-ca-gv24m","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal","openshift-multus/multus-additional-cni-plugins-7zhjr","openshift-multus/multus-mj7l7","openshift-network-diagnostics/network-check-target-256x6","kube-system/global-pull-secret-syncer-5bbrj","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z","openshift-dns/node-resolver-cqcd8","openshift-multus/network-metrics-daemon-944xj","openshift-network-operator/iptables-alerter-txjv2","openshift-ovn-kubernetes/ovnkube-node-7wdfd"] Jul 22 02:19:20.622802 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.622780 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.625727 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.625708 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.626571 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.626550 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.626757 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.626742 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.626900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.626878 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ddlq6\"" Jul 22 02:19:20.626900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.626899 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 22 02:19:20.627079 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.626899 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 22 02:19:20.628449 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.628432 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.629516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.629493 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.629633 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.629530 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-4fwmz\"" Jul 22 02:19:20.629692 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.629667 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.631273 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.631251 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.632490 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.632205 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-p7f5f\"" Jul 22 02:19:20.632490 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.632423 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.632945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.632927 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 22 02:19:20.633238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.633220 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.635497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.635460 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 22 02:19:20.636395 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.636373 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.636620 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.636586 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.636620 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.636603 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ll872\"" Jul 22 02:19:20.636814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.636795 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.639010 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.638993 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.641664 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.641642 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.641765 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.641715 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:20.644309 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.644289 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.644709 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.644690 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.645039 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645021 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-kubelet\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645129 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645055 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-var-lib-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645129 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645083 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-etc-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645129 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645108 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-systemd\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645131 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/708a0732-481b-42fa-af89-ee23dfedff12-serviceca\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645172 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-systemd-units\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645207 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-netns\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645236 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-node-log\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f01f5c-b54c-4b97-8af8-e59860e23e78-tmp-dir\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.645304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645291 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-k8s-cni-cncf-io\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645330 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-conf\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645361 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-modprobe-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645383 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysconfig\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645432 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-var-lib-kubelet\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645459 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb6wm\" (UniqueName: \"kubernetes.io/projected/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-kube-api-access-zb6wm\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645478 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/708a0732-481b-42fa-af89-ee23dfedff12-host\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645526 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-ovn\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645563 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-netns\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-multus\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645622 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645641 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-tmp\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645712 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-h2zsd\"" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-conf-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-host\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645764 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7v29\" (UniqueName: \"kubernetes.io/projected/708a0732-481b-42fa-af89-ee23dfedff12-kube-api-access-v7v29\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-slash\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.645812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645809 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-systemd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645831 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645884 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645836 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-qcdzd\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-system-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-os-release\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.645995 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-hostroot\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646020 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkhdj\" (UniqueName: \"kubernetes.io/projected/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-kube-api-access-rkhdj\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-bin\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646069 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cni-binary-copy\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646117 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-multus-certs\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61f01f5c-b54c-4b97-8af8-e59860e23e78-hosts-file\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646188 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-tuned\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646224 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-netd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646262 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646282 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646317 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-run\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646332 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646342 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-lib-modules\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646351 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 22 02:19:20.646407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646365 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646389 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-log-socket\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646415 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-bin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646286 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646449 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-kubelet\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646506 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/61f01f5c-b54c-4b97-8af8-e59860e23e78-kube-api-access-pc7zm\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646535 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646562 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-sys\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cnibin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646629 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-daemon-config\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646665 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-kubernetes\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646692 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-config\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-env-overrides\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646750 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-socket-dir-parent\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646778 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646806 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-script-lib\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646831 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wgqf\" (UniqueName: \"kubernetes.io/projected/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-kube-api-access-4wgqf\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.647286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.646865 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-etc-kubernetes\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.648030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.647366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:20.648030 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.647457 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:20.648030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.647714 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.648030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.647844 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.648370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.648350 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-zd75h\"" Jul 22 02:19:20.648486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.648381 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 22 02:19:20.649974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.649954 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.653455 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.651510 2570 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 22 02:19:20.653455 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.652510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.653974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.653955 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 22 02:19:20.654063 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.653998 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dbj7m\"" Jul 22 02:19:20.654240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.654160 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 22 02:19:20.654318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.654280 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 22 02:19:20.656305 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.656288 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b4x5k\"" Jul 22 02:19:20.656421 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.656348 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 22 02:19:20.656421 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.656349 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 22 02:19:20.656633 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.656614 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.656724 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.656690 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:20.692515 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.692492 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rnmqc" Jul 22 02:19:20.707237 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.707211 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rnmqc" Jul 22 02:19:20.733832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.733806 2570 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747061 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747101 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-tmp\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747127 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-conf-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747190 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-conf-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747289 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747331 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-host\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747354 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7v29\" (UniqueName: \"kubernetes.io/projected/708a0732-481b-42fa-af89-ee23dfedff12-kube-api-access-v7v29\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747373 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-slash\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747390 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-systemd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747416 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-cnibin\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747434 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747452 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-system-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747469 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-os-release\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747484 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-hostroot\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747482 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747509 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkhdj\" (UniqueName: \"kubernetes.io/projected/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-kube-api-access-rkhdj\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747526 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-bin\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.748329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747541 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747557 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52a3046d-e6a6-472d-a603-e6143318ce26-konnectivity-ca\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cni-binary-copy\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-multus-certs\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747613 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61f01f5c-b54c-4b97-8af8-e59860e23e78-hosts-file\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747631 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-tuned\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747646 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-kubelet-config\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747663 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747686 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-netd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747703 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747718 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-system-cni-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747734 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-dbus\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747754 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-run\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747793 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-lib-modules\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747810 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747830 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-log-socket\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.749470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747855 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tgzx\" (UniqueName: \"kubernetes.io/projected/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kube-api-access-5tgzx\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747882 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-bin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747917 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-kubelet\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747939 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/61f01f5c-b54c-4b97-8af8-e59860e23e78-kube-api-access-pc7zm\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747955 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.747984 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-sys\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c6fc704-6ef2-4b8a-84a6-723023528c98-iptables-alerter-script\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748035 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-socket-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748053 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cnibin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748099 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-daemon-config\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-host\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748130 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-bin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-run\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748241 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-kubernetes\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748263 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cnibin\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748276 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-config\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748276 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-bin\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748310 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-log-socket\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.750381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748318 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748344 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-kubernetes\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-env-overrides\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748519 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-lib-modules\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748568 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-sys\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748593 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-slash\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748602 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-multus-certs\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748644 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-system-cni-dir\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748680 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-daemon-config\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748743 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-hostroot\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748808 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-systemd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748845 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-env-overrides\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748848 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-cni-netd\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748894 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/61f01f5c-b54c-4b97-8af8-e59860e23e78-hosts-file\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748932 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-kubelet\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748952 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-registration-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-config\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.748995 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-socket-dir-parent\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.751307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749029 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749103 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-os-release\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749108 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-multus-socket-dir-parent\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749142 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-device-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-script-lib\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749180 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749248 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wgqf\" (UniqueName: \"kubernetes.io/projected/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-kube-api-access-4wgqf\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749274 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-os-release\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-cni-binary-copy\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749299 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-etc-kubernetes\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749369 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-etc-kubernetes\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749482 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749504 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-kubelet\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749535 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-kubelet\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749559 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-var-lib-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749584 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-var-lib-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.752250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-etc-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749662 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c6fc704-6ef2-4b8a-84a6-723023528c98-host-slash\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749692 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bffh8\" (UniqueName: \"kubernetes.io/projected/7c6fc704-6ef2-4b8a-84a6-723023528c98-kube-api-access-bffh8\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749711 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-etc-openvswitch\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749721 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52a3046d-e6a6-472d-a603-e6143318ce26-agent-certs\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749758 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-systemd\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749776 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovnkube-script-lib\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/708a0732-481b-42fa-af89-ee23dfedff12-serviceca\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749817 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-systemd-units\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749845 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-netns\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749873 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-node-log\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749900 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749908 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-systemd-units\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749928 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749978 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-sys-fs\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-netns\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.749987 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-systemd\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f01f5c-b54c-4b97-8af8-e59860e23e78-tmp-dir\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750034 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-node-log\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750048 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5zrz5\" (UniqueName: \"kubernetes.io/projected/bcc7da07-3195-4440-8bd5-68f8b8684573-kube-api-access-5zrz5\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750182 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-k8s-cni-cncf-io\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750199 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/708a0732-481b-42fa-af89-ee23dfedff12-serviceca\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750232 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-conf\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750257 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-k8s-cni-cncf-io\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750303 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-modprobe-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750370 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysconfig\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750385 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/61f01f5c-b54c-4b97-8af8-e59860e23e78-tmp-dir\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750407 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysctl-conf\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750408 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-var-lib-kubelet\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750423 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-sysconfig\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750451 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-var-lib-kubelet\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750460 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb6wm\" (UniqueName: \"kubernetes.io/projected/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-kube-api-access-zb6wm\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.753943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/708a0732-481b-42fa-af89-ee23dfedff12-host\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750499 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-modprobe-d\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750512 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-ovn\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750549 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-netns\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/708a0732-481b-42fa-af89-ee23dfedff12-host\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750605 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-run-ovn\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750626 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-multus\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750641 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-run-netns\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750675 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rztf\" (UniqueName: \"kubernetes.io/projected/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-kube-api-access-6rztf\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750748 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-host-var-lib-cni-multus\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.750768 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-host-run-ovn-kubernetes\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.751582 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-ovn-node-metrics-cert\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.751597 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-tmp\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.754796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.751837 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-etc-tuned\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.760054 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.760026 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkhdj\" (UniqueName: \"kubernetes.io/projected/17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2-kube-api-access-rkhdj\") pod \"multus-mj7l7\" (UID: \"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2\") " pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.760201 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.760158 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pc7zm\" (UniqueName: \"kubernetes.io/projected/61f01f5c-b54c-4b97-8af8-e59860e23e78-kube-api-access-pc7zm\") pod \"node-resolver-cqcd8\" (UID: \"61f01f5c-b54c-4b97-8af8-e59860e23e78\") " pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.760765 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.760720 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wgqf\" (UniqueName: \"kubernetes.io/projected/0bce0e38-56ae-41d0-82c4-2f22e597c6d8-kube-api-access-4wgqf\") pod \"ovnkube-node-7wdfd\" (UID: \"0bce0e38-56ae-41d0-82c4-2f22e597c6d8\") " pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.761136 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.761117 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb6wm\" (UniqueName: \"kubernetes.io/projected/03b4b0b2-5d1a-4efd-8da3-40f7460459fc-kube-api-access-zb6wm\") pod \"tuned-7fcfs\" (UID: \"03b4b0b2-5d1a-4efd-8da3-40f7460459fc\") " pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.761353 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.761335 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7v29\" (UniqueName: \"kubernetes.io/projected/708a0732-481b-42fa-af89-ee23dfedff12-kube-api-access-v7v29\") pod \"node-ca-gv24m\" (UID: \"708a0732-481b-42fa-af89-ee23dfedff12\") " pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.851389 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851351 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851402 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-sys-fs\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5zrz5\" (UniqueName: \"kubernetes.io/projected/bcc7da07-3195-4440-8bd5-68f8b8684573-kube-api-access-5zrz5\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851478 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851505 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851532 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-sys-fs\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.851577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rztf\" (UniqueName: \"kubernetes.io/projected/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-kube-api-access-6rztf\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-cnibin\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851617 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851651 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851676 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52a3046d-e6a6-472d-a603-e6143318ce26-konnectivity-ca\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.851622 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851764 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851770 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-kubelet-config\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851710 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.851814 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:21.351767378 +0000 UTC m=+3.129843326 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:20.851847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851706 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-kubelet-config\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851851 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851880 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-system-cni-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851902 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-dbus\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851926 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851956 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tgzx\" (UniqueName: \"kubernetes.io/projected/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kube-api-access-5tgzx\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.851987 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c6fc704-6ef2-4b8a-84a6-723023528c98-iptables-alerter-script\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-socket-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852042 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-registration-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852046 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-cnibin\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852070 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-device-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852073 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852101 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-os-release\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852118 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852127 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852203 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-system-cni-dir\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.852234 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852267 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-registration-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.852297 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:21.352278646 +0000 UTC m=+3.130354567 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852324 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bcc7da07-3195-4440-8bd5-68f8b8684573-cni-binary-copy\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852343 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/5f68fb46-7000-44a1-af02-354bd9b17c4a-dbus\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852372 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c6fc704-6ef2-4b8a-84a6-723023528c98-host-slash\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852379 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bcc7da07-3195-4440-8bd5-68f8b8684573-os-release\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852371 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-socket-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852381 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/52a3046d-e6a6-472d-a603-e6143318ce26-konnectivity-ca\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852414 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bffh8\" (UniqueName: \"kubernetes.io/projected/7c6fc704-6ef2-4b8a-84a6-723023528c98-kube-api-access-bffh8\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852421 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-device-dir\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852442 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7c6fc704-6ef2-4b8a-84a6-723023528c98-host-slash\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852445 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-etc-selinux\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.852660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852486 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52a3046d-e6a6-472d-a603-e6143318ce26-agent-certs\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.853227 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.852826 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7c6fc704-6ef2-4b8a-84a6-723023528c98-iptables-alerter-script\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.855197 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.855174 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/52a3046d-e6a6-472d-a603-e6143318ce26-agent-certs\") pod \"konnectivity-agent-f6xrz\" (UID: \"52a3046d-e6a6-472d-a603-e6143318ce26\") " pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:20.881038 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.880960 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:20.881038 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.880988 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:20.881038 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.881004 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:20.881326 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:20.881082 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:21.381061392 +0000 UTC m=+3.159137314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:20.882522 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.882497 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tgzx\" (UniqueName: \"kubernetes.io/projected/8f0f9a24-51c1-405b-bb68-fa4d0970e86c-kube-api-access-5tgzx\") pod \"aws-ebs-csi-driver-node-hh88z\" (UID: \"8f0f9a24-51c1-405b-bb68-fa4d0970e86c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:20.882632 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.882502 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bffh8\" (UniqueName: \"kubernetes.io/projected/7c6fc704-6ef2-4b8a-84a6-723023528c98-kube-api-access-bffh8\") pod \"iptables-alerter-txjv2\" (UID: \"7c6fc704-6ef2-4b8a-84a6-723023528c98\") " pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.882824 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.882803 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rztf\" (UniqueName: \"kubernetes.io/projected/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-kube-api-access-6rztf\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:20.883488 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.883468 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5zrz5\" (UniqueName: \"kubernetes.io/projected/bcc7da07-3195-4440-8bd5-68f8b8684573-kube-api-access-5zrz5\") pod \"multus-additional-cni-plugins-7zhjr\" (UID: \"bcc7da07-3195-4440-8bd5-68f8b8684573\") " pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.940711 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.940674 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mj7l7" Jul 22 02:19:20.947694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.947673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-cqcd8" Jul 22 02:19:20.959307 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.959283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" Jul 22 02:19:20.965222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.965194 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gv24m" Jul 22 02:19:20.972784 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.972766 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:20.982537 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.982518 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" Jul 22 02:19:20.990349 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.990330 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-txjv2" Jul 22 02:19:20.999274 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:20.999252 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" Jul 22 02:19:21.004324 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.004295 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:21.357661 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.357564 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:21.357661 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.357635 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:21.357893 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.357736 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:21.357893 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.357772 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:21.357893 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.357806 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:22.357785657 +0000 UTC m=+4.135861582 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:21.357893 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.357824 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:22.357817038 +0000 UTC m=+4.135892958 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:21.459016 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.458975 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:21.459187 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.459131 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:21.459187 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.459145 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:21.459187 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.459155 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:21.459307 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:21.459229 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:22.459210987 +0000 UTC m=+4.237286908 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:21.710359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.710257 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:21 +0000 UTC" deadline="2027-12-23 22:57:26.038685088 +0000 UTC" Jul 22 02:19:21.710359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:21.710308 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12476h38m4.328379557s" Jul 22 02:19:21.804352 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.804303 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bce0e38_56ae_41d0_82c4_2f22e597c6d8.slice/crio-f0a75be8f96a65f955681cfc31138f14aa5cdcc6bc82b892766de0832ac73769 WatchSource:0}: Error finding container f0a75be8f96a65f955681cfc31138f14aa5cdcc6bc82b892766de0832ac73769: Status 404 returned error can't find the container with id f0a75be8f96a65f955681cfc31138f14aa5cdcc6bc82b892766de0832ac73769 Jul 22 02:19:21.813280 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.813251 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod708a0732_481b_42fa_af89_ee23dfedff12.slice/crio-05883f47cb2f514debbed51c4187081527ab45c1e3f0690986d0ed459891911a WatchSource:0}: Error finding container 05883f47cb2f514debbed51c4187081527ab45c1e3f0690986d0ed459891911a: Status 404 returned error can't find the container with id 05883f47cb2f514debbed51c4187081527ab45c1e3f0690986d0ed459891911a Jul 22 02:19:21.814067 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.814044 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61f01f5c_b54c_4b97_8af8_e59860e23e78.slice/crio-ea8de874c441c5d5daaf4f813567771d00fb09ec783367ed519181dfa3639d01 WatchSource:0}: Error finding container ea8de874c441c5d5daaf4f813567771d00fb09ec783367ed519181dfa3639d01: Status 404 returned error can't find the container with id ea8de874c441c5d5daaf4f813567771d00fb09ec783367ed519181dfa3639d01 Jul 22 02:19:21.814802 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.814766 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03b4b0b2_5d1a_4efd_8da3_40f7460459fc.slice/crio-0cb42ba2977529ab649e75e18a06c7deb50ff850e635076a63b4640d4b79a570 WatchSource:0}: Error finding container 0cb42ba2977529ab649e75e18a06c7deb50ff850e635076a63b4640d4b79a570: Status 404 returned error can't find the container with id 0cb42ba2977529ab649e75e18a06c7deb50ff850e635076a63b4640d4b79a570 Jul 22 02:19:21.837436 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.837414 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b9d2f8_6567_4147_87fd_f8ef4d9f4cc2.slice/crio-19fd02b20916b1fef48511f3bf46784482a37aed496bb3b644eb772206ea522f WatchSource:0}: Error finding container 19fd02b20916b1fef48511f3bf46784482a37aed496bb3b644eb772206ea522f: Status 404 returned error can't find the container with id 19fd02b20916b1fef48511f3bf46784482a37aed496bb3b644eb772206ea522f Jul 22 02:19:21.838719 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.838698 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c6fc704_6ef2_4b8a_84a6_723023528c98.slice/crio-356b0cd651692e0940867477a91f7553f7a550474cb8874b553a6a6ef9b25100 WatchSource:0}: Error finding container 356b0cd651692e0940867477a91f7553f7a550474cb8874b553a6a6ef9b25100: Status 404 returned error can't find the container with id 356b0cd651692e0940867477a91f7553f7a550474cb8874b553a6a6ef9b25100 Jul 22 02:19:21.840087 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.840062 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcc7da07_3195_4440_8bd5_68f8b8684573.slice/crio-c13788e4c3f5ffd3d094149d3ee28ebf16774f25acec8cc17493e5ae2fc57bc0 WatchSource:0}: Error finding container c13788e4c3f5ffd3d094149d3ee28ebf16774f25acec8cc17493e5ae2fc57bc0: Status 404 returned error can't find the container with id c13788e4c3f5ffd3d094149d3ee28ebf16774f25acec8cc17493e5ae2fc57bc0 Jul 22 02:19:21.841097 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.841074 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52a3046d_e6a6_472d_a603_e6143318ce26.slice/crio-566927207ec368cc5355a1cb070e6cc11fa974e5105ba3590b2a41b25fddde10 WatchSource:0}: Error finding container 566927207ec368cc5355a1cb070e6cc11fa974e5105ba3590b2a41b25fddde10: Status 404 returned error can't find the container with id 566927207ec368cc5355a1cb070e6cc11fa974e5105ba3590b2a41b25fddde10 Jul 22 02:19:21.841972 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:21.841794 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f0f9a24_51c1_405b_bb68_fa4d0970e86c.slice/crio-8e5b16191c4b081b86cf0b00f37f5ad44b9fd2bfad95e1fa22ef19d89606663e WatchSource:0}: Error finding container 8e5b16191c4b081b86cf0b00f37f5ad44b9fd2bfad95e1fa22ef19d89606663e: Status 404 returned error can't find the container with id 8e5b16191c4b081b86cf0b00f37f5ad44b9fd2bfad95e1fa22ef19d89606663e Jul 22 02:19:22.366574 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.366239 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:22.366574 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.366537 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:22.366813 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.366606 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:22.366813 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.366664 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:24.366647009 +0000 UTC m=+6.144722930 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:22.366813 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.366743 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:22.366813 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.366780 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:24.36677088 +0000 UTC m=+6.144846801 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:22.467268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.467228 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:22.467631 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.467506 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:22.467631 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.467533 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:22.467631 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.467568 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:22.467811 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.467638 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:24.467614815 +0000 UTC m=+6.245690761 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:22.710744 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.710614 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-21 02:14:21 +0000 UTC" deadline="2028-04-03 23:28:12.179298708 +0000 UTC" Jul 22 02:19:22.710744 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.710650 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14925h8m49.468653919s" Jul 22 02:19:22.746983 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.746405 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:22.746983 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.746565 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:22.746983 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.746898 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:22.747311 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.747005 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:22.747311 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.747061 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:22.747311 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:22.747139 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:22.776536 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.775863 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" event={"ID":"e7e85b7071d89f6d9d1647bbf823b423","Type":"ContainerStarted","Data":"fd500a797a3a0abdf8a128cde0db2ce38269e398ae5711fe3ac1c2d826b3259f"} Jul 22 02:19:22.789861 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.789753 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" event={"ID":"8f0f9a24-51c1-405b-bb68-fa4d0970e86c","Type":"ContainerStarted","Data":"8e5b16191c4b081b86cf0b00f37f5ad44b9fd2bfad95e1fa22ef19d89606663e"} Jul 22 02:19:22.801299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.801227 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-149.ec2.internal" podStartSLOduration=2.80119144 podStartE2EDuration="2.80119144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:22.799374783 +0000 UTC m=+4.577450729" watchObservedRunningTime="2026-07-22 02:19:22.80119144 +0000 UTC m=+4.579267383" Jul 22 02:19:22.804112 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.804080 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cqcd8" event={"ID":"61f01f5c-b54c-4b97-8af8-e59860e23e78","Type":"ContainerStarted","Data":"ea8de874c441c5d5daaf4f813567771d00fb09ec783367ed519181dfa3639d01"} Jul 22 02:19:22.816252 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.816224 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mj7l7" event={"ID":"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2","Type":"ContainerStarted","Data":"19fd02b20916b1fef48511f3bf46784482a37aed496bb3b644eb772206ea522f"} Jul 22 02:19:22.824142 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.824109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerStarted","Data":"c13788e4c3f5ffd3d094149d3ee28ebf16774f25acec8cc17493e5ae2fc57bc0"} Jul 22 02:19:22.828837 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.828806 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6xrz" event={"ID":"52a3046d-e6a6-472d-a603-e6143318ce26","Type":"ContainerStarted","Data":"566927207ec368cc5355a1cb070e6cc11fa974e5105ba3590b2a41b25fddde10"} Jul 22 02:19:22.836288 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.836256 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-txjv2" event={"ID":"7c6fc704-6ef2-4b8a-84a6-723023528c98","Type":"ContainerStarted","Data":"356b0cd651692e0940867477a91f7553f7a550474cb8874b553a6a6ef9b25100"} Jul 22 02:19:22.843366 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.843326 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gv24m" event={"ID":"708a0732-481b-42fa-af89-ee23dfedff12","Type":"ContainerStarted","Data":"05883f47cb2f514debbed51c4187081527ab45c1e3f0690986d0ed459891911a"} Jul 22 02:19:22.852030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.851997 2570 generic.go:356] "Generic (PLEG): container finished" podID="050ebd4d45e7f09a481ee99742944a20" containerID="1f00af18ebda7250d5569fabff7e4f3d0c466d073e8b7057656dadaf8858ff9e" exitCode=0 Jul 22 02:19:22.852134 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.852096 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" event={"ID":"050ebd4d45e7f09a481ee99742944a20","Type":"ContainerDied","Data":"1f00af18ebda7250d5569fabff7e4f3d0c466d073e8b7057656dadaf8858ff9e"} Jul 22 02:19:22.867745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.867710 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" event={"ID":"03b4b0b2-5d1a-4efd-8da3-40f7460459fc","Type":"ContainerStarted","Data":"0cb42ba2977529ab649e75e18a06c7deb50ff850e635076a63b4640d4b79a570"} Jul 22 02:19:22.876231 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:22.876189 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"f0a75be8f96a65f955681cfc31138f14aa5cdcc6bc82b892766de0832ac73769"} Jul 22 02:19:23.897855 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:23.897780 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" event={"ID":"050ebd4d45e7f09a481ee99742944a20","Type":"ContainerStarted","Data":"85a8ba0cd6d430bd4826ca90b9363d86edd34764fdff874936829d608d703a71"} Jul 22 02:19:23.920093 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:23.919547 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-149.ec2.internal" podStartSLOduration=3.919524156 podStartE2EDuration="3.919524156s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:23.918622383 +0000 UTC m=+5.696698330" watchObservedRunningTime="2026-07-22 02:19:23.919524156 +0000 UTC m=+5.697600102" Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.383193 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.383305 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.383472 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.383534 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:28.383516917 +0000 UTC m=+10.161592843 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.383935 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:24.384126 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.383986 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:28.383970181 +0000 UTC m=+10.162046107 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:24.484165 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.484047 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:24.484337 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.484243 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:24.484337 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.484280 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:24.484337 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.484294 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:24.484506 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.484359 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:28.484337597 +0000 UTC m=+10.262413520 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:24.744489 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.744333 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:24.744489 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.744472 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:24.744941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.744763 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:24.744941 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.744884 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:24.744941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:24.744914 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:24.745163 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:24.745027 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:26.743290 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:26.743257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:26.743851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:26.743302 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:26.743851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:26.743257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:26.743851 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:26.743429 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:26.743851 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:26.743504 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:26.743851 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:26.743602 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:28.426301 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.426257 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:28.426673 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.426323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:28.426673 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.426455 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:28.426673 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.426514 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:36.42650004 +0000 UTC m=+18.204575959 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:28.426876 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.426856 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:28.426990 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.426897 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:36.426885861 +0000 UTC m=+18.204961780 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:28.527064 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.527012 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:28.527260 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.527228 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:28.527260 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.527248 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:28.527260 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.527261 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:28.527420 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.527330 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:36.527311098 +0000 UTC m=+18.305387039 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:28.743562 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.743529 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:28.743769 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.743539 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:28.743769 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.743650 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:28.743769 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.743714 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:28.743929 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:28.743784 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:28.743929 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:28.743854 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:30.742706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:30.742675 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:30.743221 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:30.742876 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:30.743304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:30.743268 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:30.747670 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:30.747573 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:30.748653 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:30.748448 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:30.748653 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:30.748543 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:32.742677 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:32.742635 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:32.742677 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:32.742635 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:32.743245 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:32.742650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:32.743245 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:32.742768 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:32.743245 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:32.742847 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:32.743245 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:32.742924 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:34.742516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:34.742478 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:34.742516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:34.742509 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:34.743068 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:34.742480 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:34.743068 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:34.742625 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:34.743068 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:34.742700 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:34.743068 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:34.742799 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:36.490049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.490015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:36.490433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.490080 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:36.490433 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.490178 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:36.490433 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.490178 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:36.490433 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.490230 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:19:52.49021393 +0000 UTC m=+34.268289848 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:36.490433 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.490247 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:19:52.490238204 +0000 UTC m=+34.268314122 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:36.590908 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.590878 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:36.591036 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.591023 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:36.591084 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.591040 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:36.591084 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.591049 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:36.591185 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.591173 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:52.591156859 +0000 UTC m=+34.369232780 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:36.743010 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.742931 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:36.743152 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.742932 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:36.743152 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.743040 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:36.743226 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:36.742947 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:36.743226 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.743192 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:36.743304 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:36.743283 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:38.744405 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:38.744363 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:38.744842 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:38.744482 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:38.744842 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:38.744532 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:38.744842 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:38.744625 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:38.744842 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:38.744796 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:38.745063 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:38.744897 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:40.743057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:40.743024 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:40.743057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:40.743045 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:40.743601 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:40.743166 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:40.743601 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:40.743215 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:40.743601 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:40.743346 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:40.743601 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:40.743435 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:41.938116 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.937969 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="fa79a1c1795078e21f7622b2523c8ff4f9a8483bd8317f22a5c5751fcec53d6f" exitCode=0 Jul 22 02:19:41.938775 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.938048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"fa79a1c1795078e21f7622b2523c8ff4f9a8483bd8317f22a5c5751fcec53d6f"} Jul 22 02:19:41.939752 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.939732 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-f6xrz" event={"ID":"52a3046d-e6a6-472d-a603-e6143318ce26","Type":"ContainerStarted","Data":"2d0c4d19b6c5312bab7b57dacecd8d5d8fe34411b5a26c4a89edad1888cf3ad9"} Jul 22 02:19:41.941168 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.941112 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gv24m" event={"ID":"708a0732-481b-42fa-af89-ee23dfedff12","Type":"ContainerStarted","Data":"f52d4444a135b4f1f6aa8775ff8aa273e76d77362ee9976c458923f85015695c"} Jul 22 02:19:41.942588 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.942566 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" event={"ID":"03b4b0b2-5d1a-4efd-8da3-40f7460459fc","Type":"ContainerStarted","Data":"2eafff54e1dc494e81ae4e8062bde0f50808ad2eafc9423129a9fc594431982c"} Jul 22 02:19:41.945656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.945620 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"44551185761b6e366dfccdc512c12e53bacb3e7c649d025e90edd41f31d21e61"} Jul 22 02:19:41.945770 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.945755 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"f17cde2e24ac19e5985d62fa89c53c774fdda24f334a5e2a507ee5c06ea858f9"} Jul 22 02:19:41.947531 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.947503 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" event={"ID":"8f0f9a24-51c1-405b-bb68-fa4d0970e86c","Type":"ContainerStarted","Data":"0867a156857f1be32665de1178f245cd7eaf63a6e48913a50d6b475bbd364d02"} Jul 22 02:19:41.949090 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.949066 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-cqcd8" event={"ID":"61f01f5c-b54c-4b97-8af8-e59860e23e78","Type":"ContainerStarted","Data":"5119d83d112be46fc6f7b2383bf2b368e2b8c982b1b59108269ee5a2d9f5b7c0"} Jul 22 02:19:41.950343 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.950324 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mj7l7" event={"ID":"17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2","Type":"ContainerStarted","Data":"5b40b6a88c89d411486690c6030b36f23ed178cb3ba83cfc2ac41e31ede917db"} Jul 22 02:19:41.975616 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.975577 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-cqcd8" podStartSLOduration=3.587397556 podStartE2EDuration="22.975563787s" totalImagesPullingTime="19.388166231s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:41.974893861 +0000 UTC m=+23.752969801" watchObservedRunningTime="2026-07-22 02:19:41.975563787 +0000 UTC m=+23.753639727" Jul 22 02:19:41.992244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:41.992204 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7fcfs" podStartSLOduration=3.546456655 podStartE2EDuration="22.992122779s" totalImagesPullingTime="19.445666124s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:41.991754529 +0000 UTC m=+23.769830471" watchObservedRunningTime="2026-07-22 02:19:41.992122779 +0000 UTC m=+23.770198720" Jul 22 02:19:42.005635 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.005595 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gv24m" podStartSLOduration=3.892083538 podStartE2EDuration="23.005580107s" totalImagesPullingTime="19.113496569s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:42.005365902 +0000 UTC m=+23.783441844" watchObservedRunningTime="2026-07-22 02:19:42.005580107 +0000 UTC m=+23.783656048" Jul 22 02:19:42.036837 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.036675 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mj7l7" podStartSLOduration=3.5780994 podStartE2EDuration="23.03666109s" totalImagesPullingTime="19.45856169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:42.036231631 +0000 UTC m=+23.814307572" watchObservedRunningTime="2026-07-22 02:19:42.03666109 +0000 UTC m=+23.814737030" Jul 22 02:19:42.721492 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.721463 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 22 02:19:42.743180 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.743157 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:42.743276 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.743159 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:42.743386 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:42.743269 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:42.743386 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.743164 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:42.743386 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:42.743359 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:42.743504 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:42.743407 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:42.954461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.954423 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-txjv2" event={"ID":"7c6fc704-6ef2-4b8a-84a6-723023528c98","Type":"ContainerStarted","Data":"d83c92ab839bc30d6718a1ad9b6a09737ee17d21f407a55ba2eda0896cc09039"} Jul 22 02:19:42.957776 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.957757 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:19:42.958153 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958107 2570 generic.go:356] "Generic (PLEG): container finished" podID="0bce0e38-56ae-41d0-82c4-2f22e597c6d8" containerID="44551185761b6e366dfccdc512c12e53bacb3e7c649d025e90edd41f31d21e61" exitCode=1 Jul 22 02:19:42.958232 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958176 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerDied","Data":"44551185761b6e366dfccdc512c12e53bacb3e7c649d025e90edd41f31d21e61"} Jul 22 02:19:42.958289 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958262 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"035e0c288978ffa495b5c08481adb81882b885b37229e4cbccdf59feb1243a4c"} Jul 22 02:19:42.958342 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958315 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"fc6e3a449e2b699d64ce0f8ad7f0bcfe7cf218ea353b1659a775f6cda4d16cb9"} Jul 22 02:19:42.958400 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958365 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"48f776d98c06ff680c894ce614b20e83303572e520bab39b437c3638ebd5f8f5"} Jul 22 02:19:42.958456 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.958413 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"ff54074977631708b5e1c14a8af4e3d01723216d6559a081fcb1e87dd5fdd501"} Jul 22 02:19:42.959738 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.959718 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" event={"ID":"8f0f9a24-51c1-405b-bb68-fa4d0970e86c","Type":"ContainerStarted","Data":"7fb948e39d0150f612f05c04bc2c7b44c648413d3d800a7657c108aa05fefb66"} Jul 22 02:19:42.969560 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.969525 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-txjv2" podStartSLOduration=4.586461617 podStartE2EDuration="23.969514575s" totalImagesPullingTime="19.383052958s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:42.969295196 +0000 UTC m=+24.747371137" watchObservedRunningTime="2026-07-22 02:19:42.969514575 +0000 UTC m=+24.747590515" Jul 22 02:19:42.969721 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:42.969703 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-f6xrz" podStartSLOduration=13.046229231 podStartE2EDuration="23.969696629s" totalImagesPullingTime="10.923467398s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:42.054731051 +0000 UTC m=+23.832806993" watchObservedRunningTime="2026-07-22 02:19:42.969696629 +0000 UTC m=+24.747772570" Jul 22 02:19:43.468360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.468329 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:43.469127 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.469108 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:43.712368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.712275 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-22T02:19:42.721488265Z","UUID":"8067a7d3-9aaa-4685-a451-6faa26c8baaa","Handler":null,"Name":"","Endpoint":""} Jul 22 02:19:43.713830 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.713811 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 22 02:19:43.713949 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.713854 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 22 02:19:43.964939 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.964899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" event={"ID":"8f0f9a24-51c1-405b-bb68-fa4d0970e86c","Type":"ContainerStarted","Data":"f83c23055f9f6747423d46d2b36d290749afa9f8e8ae02c880d34043e8187ddc"} Jul 22 02:19:43.965605 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.965580 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:43.965705 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.965683 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-f6xrz" Jul 22 02:19:43.984774 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:43.984684 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-hh88z" podStartSLOduration=3.8126577900000003 podStartE2EDuration="24.984664822s" totalImagesPullingTime="21.172007032s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:43.984379489 +0000 UTC m=+25.762455430" watchObservedRunningTime="2026-07-22 02:19:43.984664822 +0000 UTC m=+25.762740765" Jul 22 02:19:44.743597 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:44.743409 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:44.743766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:44.743409 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:44.743766 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:44.743678 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:44.743766 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:44.743745 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:44.743766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:44.743472 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:44.743896 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:44.743810 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:44.970998 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:44.970973 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:19:44.971909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:44.971878 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"50aa1f5cdbc71a7a85d52cfbae9607e0fbda4fa829604ade496b9bddfa692261"} Jul 22 02:19:46.743119 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.742923 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:46.744031 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:46.743206 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:46.744031 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.742925 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:46.744031 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.742956 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:46.744031 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:46.743324 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:46.744031 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:46.743378 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:46.979366 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.979338 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:19:46.979747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.979721 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"5f106b8f0f6eebdcd0e7630ff39661840d5511d187621aa1078103c8419bb2f9"} Jul 22 02:19:46.979981 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.979963 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:46.980074 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.980040 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:46.980148 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.980116 2570 scope.go:117] "RemoveContainer" containerID="44551185761b6e366dfccdc512c12e53bacb3e7c649d025e90edd41f31d21e61" Jul 22 02:19:46.981757 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.981732 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="02430267ee00b48865166a4292cf823907818fa48ea1131c261ba1789256fd0a" exitCode=0 Jul 22 02:19:46.981845 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.981781 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"02430267ee00b48865166a4292cf823907818fa48ea1131c261ba1789256fd0a"} Jul 22 02:19:46.997570 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:46.997554 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:47.986057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:47.985977 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="57e4db8440d56eddffce27932b39e857b9ec29a974fcfef1fbc761edbef2a40c" exitCode=0 Jul 22 02:19:47.986508 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:47.986064 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"57e4db8440d56eddffce27932b39e857b9ec29a974fcfef1fbc761edbef2a40c"} Jul 22 02:19:47.989645 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:47.989628 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:19:47.990418 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:47.990394 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" event={"ID":"0bce0e38-56ae-41d0-82c4-2f22e597c6d8","Type":"ContainerStarted","Data":"c5a32dc04e37a449e70965b8e9074e1ccb8565f3c5d01dfff44ab8e89d76b5a6"} Jul 22 02:19:47.990582 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:47.990568 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:48.005731 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.005711 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:19:48.040542 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.040497 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" podStartSLOduration=9.104402703 podStartE2EDuration="29.040459098s" totalImagesPullingTime="19.936056395s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:48.03867366 +0000 UTC m=+29.816749611" watchObservedRunningTime="2026-07-22 02:19:48.040459098 +0000 UTC m=+29.818535066" Jul 22 02:19:48.744017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.743994 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:48.744111 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.744093 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:48.744205 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.744101 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:48.744205 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.744189 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:48.744308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.744207 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:48.744308 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.744260 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:48.824757 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.824576 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5bbrj"] Jul 22 02:19:48.827863 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.827841 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-944xj"] Jul 22 02:19:48.828389 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.828369 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-256x6"] Jul 22 02:19:48.995414 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.995385 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="adff34766ebc96799e97e7e11337d76113df06e1d6b12fed219c5c46ec8e4e60" exitCode=0 Jul 22 02:19:48.995814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.995459 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:48.995814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.995482 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"adff34766ebc96799e97e7e11337d76113df06e1d6b12fed219c5c46ec8e4e60"} Jul 22 02:19:48.995814 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.995760 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:48.996125 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.996109 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:48.996269 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.996217 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:48.996269 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:48.996235 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:48.996344 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:48.996317 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:50.742860 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:50.742829 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:50.743319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:50.742829 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:50.743319 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:50.742979 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:50.743319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:50.742998 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:50.743319 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:50.743111 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:50.743319 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:50.743193 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:52.505705 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.505667 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:52.506492 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.505742 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:52.506492 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.505837 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:52.506492 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.505865 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:52.506492 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.505913 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs podName:4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f nodeName:}" failed. No retries permitted until 2026-07-22 02:20:24.505892964 +0000 UTC m=+66.283968891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs") pod "network-metrics-daemon-944xj" (UID: "4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 22 02:19:52.506492 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.505935 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret podName:5f68fb46-7000-44a1-af02-354bd9b17c4a nodeName:}" failed. No retries permitted until 2026-07-22 02:20:24.505924683 +0000 UTC m=+66.284000608 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret") pod "global-pull-secret-syncer-5bbrj" (UID: "5f68fb46-7000-44a1-af02-354bd9b17c4a") : object "kube-system"/"original-pull-secret" not registered Jul 22 02:19:52.606333 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.606297 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:52.606511 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.606470 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 22 02:19:52.606511 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.606500 2570 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 22 02:19:52.606511 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.606514 2570 projected.go:196] Error preparing data for projected volume kube-api-access-xcfl7 for pod openshift-network-diagnostics/network-check-target-256x6: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:52.606644 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.606576 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7 podName:2a0830c2-c123-43d3-a033-b6eabe699142 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:24.606559478 +0000 UTC m=+66.384635418 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xcfl7" (UniqueName: "kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7") pod "network-check-target-256x6" (UID: "2a0830c2-c123-43d3-a033-b6eabe699142") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 22 02:19:52.742481 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.742446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:52.742641 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.742446 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:52.742641 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.742574 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:52.742759 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:52.742451 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:52.742759 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.742669 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:52.742849 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:52.742759 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:54.743345 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:54.743305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:54.743345 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:54.743335 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:54.744177 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:54.743316 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:54.744177 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:54.743419 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-944xj" podUID="4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f" Jul 22 02:19:54.744177 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:54.743491 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-5bbrj" podUID="5f68fb46-7000-44a1-af02-354bd9b17c4a" Jul 22 02:19:54.744177 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:54.743573 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-256x6" podUID="2a0830c2-c123-43d3-a033-b6eabe699142" Jul 22 02:19:55.010184 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.010085 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="74482120b1af94cdd32faafa4145a4dfa5d5b2c91197b2315fcee10a56cb0e60" exitCode=0 Jul 22 02:19:55.010184 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.010164 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"74482120b1af94cdd32faafa4145a4dfa5d5b2c91197b2315fcee10a56cb0e60"} Jul 22 02:19:55.049248 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.049060 2570 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-149.ec2.internal" event="NodeReady" Jul 22 02:19:55.050925 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.049501 2570 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 22 02:19:55.089039 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.089020 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:19:55.113612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.113585 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:19:55.113612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.113618 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-8dr7b"] Jul 22 02:19:55.113786 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.113715 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.116292 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.116270 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f4l4l\"" Jul 22 02:19:55.116451 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.116271 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 22 02:19:55.116451 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.116275 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 22 02:19:55.116656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.116641 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 22 02:19:55.122691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.122665 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 22 02:19:55.133763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.133727 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4jq5p"] Jul 22 02:19:55.133877 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.133834 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.136765 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.136743 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 22 02:19:55.136871 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.136809 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k4rg\"" Jul 22 02:19:55.136987 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.136971 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 22 02:19:55.149885 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.149864 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8dr7b"] Jul 22 02:19:55.149991 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.149906 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4jq5p"] Jul 22 02:19:55.149991 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.149945 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.152633 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.152615 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 22 02:19:55.152755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.152679 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 22 02:19:55.152755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.152699 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 22 02:19:55.152755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.152710 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-ptnrq\"" Jul 22 02:19:55.225441 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225414 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225446 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zqfjx\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225469 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.225589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225485 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225502 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/32b449dc-4ee5-49e3-8543-de214f550403-config-volume\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.225589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225559 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rzh2\" (UniqueName: \"kubernetes.io/projected/32b449dc-4ee5-49e3-8543-de214f550403-kube-api-access-5rzh2\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.225750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225597 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225672 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.225750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225738 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32b449dc-4ee5-49e3-8543-de214f550403-tmp-dir\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.225893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.225753 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.326468 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326375 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rzh2\" (UniqueName: \"kubernetes.io/projected/32b449dc-4ee5-49e3-8543-de214f550403-kube-api-access-5rzh2\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.326468 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326421 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.326468 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326462 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.326712 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qxk9l\" (UniqueName: \"kubernetes.io/projected/bdae54c5-819f-46d8-80e4-0160b8ae9732-kube-api-access-qxk9l\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.326712 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326533 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.326712 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326566 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.326712 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32b449dc-4ee5-49e3-8543-de214f550403-tmp-dir\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.326899 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.326623 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.326986 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.327013 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327014 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdae54c5-819f-46d8-80e4-0160b8ae9732-cert\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327064 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.327087 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:19:55.827065579 +0000 UTC m=+37.605141500 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:19:55.327224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327205 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zqfjx\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327250 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327280 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327283 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327317 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/32b449dc-4ee5-49e3-8543-de214f550403-config-volume\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.327340 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32b449dc-4ee5-49e3-8543-de214f550403-tmp-dir\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.327572 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.327446 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:19:55.328120 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.328096 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:55.828068363 +0000 UTC m=+37.606144283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:19:55.328659 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.328587 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.328973 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.328856 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/32b449dc-4ee5-49e3-8543-de214f550403-config-volume\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.332276 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.332245 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.332349 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.332285 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.332394 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.332341 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.335472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.335446 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rzh2\" (UniqueName: \"kubernetes.io/projected/32b449dc-4ee5-49e3-8543-de214f550403-kube-api-access-5rzh2\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.338686 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.338667 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.339060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.339040 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zqfjx\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.428182 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.428122 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qxk9l\" (UniqueName: \"kubernetes.io/projected/bdae54c5-819f-46d8-80e4-0160b8ae9732-kube-api-access-qxk9l\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.428388 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.428218 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdae54c5-819f-46d8-80e4-0160b8ae9732-cert\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.430713 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.430688 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/bdae54c5-819f-46d8-80e4-0160b8ae9732-cert\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.436526 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.436507 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qxk9l\" (UniqueName: \"kubernetes.io/projected/bdae54c5-819f-46d8-80e4-0160b8ae9732-kube-api-access-qxk9l\") pod \"ingress-canary-4jq5p\" (UID: \"bdae54c5-819f-46d8-80e4-0160b8ae9732\") " pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.458585 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.458552 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4jq5p" Jul 22 02:19:55.666095 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.665914 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4jq5p"] Jul 22 02:19:55.669883 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:55.669857 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbdae54c5_819f_46d8_80e4_0160b8ae9732.slice/crio-9401124f22fa45ba0c3a9a2f0fd7c3a6336e6c5cfc36d1a1f655301df1cc09ed WatchSource:0}: Error finding container 9401124f22fa45ba0c3a9a2f0fd7c3a6336e6c5cfc36d1a1f655301df1cc09ed: Status 404 returned error can't find the container with id 9401124f22fa45ba0c3a9a2f0fd7c3a6336e6c5cfc36d1a1f655301df1cc09ed Jul 22 02:19:55.831791 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.831756 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:55.831805 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.831905 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.831915 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.831925 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.831973 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:56.831956708 +0000 UTC m=+38.610032647 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:19:55.832175 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:55.831988 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:19:56.831982056 +0000 UTC m=+38.610057975 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:19:56.017748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.017716 2570 generic.go:356] "Generic (PLEG): container finished" podID="bcc7da07-3195-4440-8bd5-68f8b8684573" containerID="49cd15a1fc849d482b4cdab488ba333b060ac8988ea1ae7202bb03668bd3361a" exitCode=0 Jul 22 02:19:56.017932 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.017799 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerDied","Data":"49cd15a1fc849d482b4cdab488ba333b060ac8988ea1ae7202bb03668bd3361a"} Jul 22 02:19:56.019489 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.019468 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4jq5p" event={"ID":"bdae54c5-819f-46d8-80e4-0160b8ae9732","Type":"ContainerStarted","Data":"9401124f22fa45ba0c3a9a2f0fd7c3a6336e6c5cfc36d1a1f655301df1cc09ed"} Jul 22 02:19:56.743498 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.743464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:19:56.743672 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.743464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:19:56.743733 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.743696 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:19:56.747398 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.747379 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 22 02:19:56.748171 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.748125 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:19:56.748171 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.748165 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 22 02:19:56.748760 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.748740 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\"" Jul 22 02:19:56.749399 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.749381 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:19:56.749484 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.749444 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\"" Jul 22 02:19:56.840441 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.840409 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:56.840472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:56.840570 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:56.840584 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:56.840608 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:56.840642 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:19:58.840621864 +0000 UTC m=+40.618697805 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:19:56.840840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:56.840658 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:58.840651822 +0000 UTC m=+40.618727741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:19:57.026062 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:57.025981 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" event={"ID":"bcc7da07-3195-4440-8bd5-68f8b8684573","Type":"ContainerStarted","Data":"8c07321657812c16882ce497b0c34aeaca64738c1959084c035cf02e085d5c18"} Jul 22 02:19:57.051458 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:57.050954 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7zhjr" podStartSLOduration=7.536105046 podStartE2EDuration="38.050935197s" totalImagesPullingTime="30.514830151s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:57.049227333 +0000 UTC m=+38.827303274" watchObservedRunningTime="2026-07-22 02:19:57.050935197 +0000 UTC m=+38.829011140" Jul 22 02:19:58.148872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.148672 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw"] Jul 22 02:19:58.159018 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.158996 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.161497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.161473 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw"] Jul 22 02:19:58.161978 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.161963 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.162035 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.161995 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 22 02:19:58.163094 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.163073 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-5fqrf\"" Jul 22 02:19:58.163094 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.163089 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.163332 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.163303 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 22 02:19:58.253722 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.253691 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0934c3d1-503f-49e4-8a19-235af2f466fb-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.253907 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.253745 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0934c3d1-503f-49e4-8a19-235af2f466fb-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.253907 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.253811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x5vx\" (UniqueName: \"kubernetes.io/projected/0934c3d1-503f-49e4-8a19-235af2f466fb-kube-api-access-8x5vx\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.255868 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.255843 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55"] Jul 22 02:19:58.268230 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.268207 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9"] Jul 22 02:19:58.268368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.268349 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" Jul 22 02:19:58.271466 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.271440 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-bqdxl\"" Jul 22 02:19:58.289306 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.289280 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc"] Jul 22 02:19:58.289429 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.289379 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.292125 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.292101 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 22 02:19:58.292125 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.292105 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mw6nm\"" Jul 22 02:19:58.292339 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.292112 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.293295 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.293272 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.293414 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.293373 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 22 02:19:58.299987 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.299968 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l4vnz"] Jul 22 02:19:58.300092 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.300077 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.302724 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.302704 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.302812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.302731 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 22 02:19:58.302812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.302777 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 22 02:19:58.302812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.302796 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-7ffmc\"" Jul 22 02:19:58.302812 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.302778 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.311509 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.311490 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-6d6ff7ddd6-r7kpn"] Jul 22 02:19:58.311601 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.311585 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.313987 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.313967 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 22 02:19:58.314098 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.314081 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 22 02:19:58.314213 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.314081 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.314213 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.314090 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.314303 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.314214 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-jqnwz\"" Jul 22 02:19:58.320382 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.319542 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 22 02:19:58.325448 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325429 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55"] Jul 22 02:19:58.325541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325460 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc"] Jul 22 02:19:58.325541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325473 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9"] Jul 22 02:19:58.325541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325484 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l4vnz"] Jul 22 02:19:58.325541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325498 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d6ff7ddd6-r7kpn"] Jul 22 02:19:58.325658 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.325540 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.327912 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.327892 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 22 02:19:58.327912 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.327905 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 22 02:19:58.328066 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.327909 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 22 02:19:58.328066 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.327902 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d655c\"" Jul 22 02:19:58.328263 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.328247 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.328528 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.328478 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 22 02:19:58.328613 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.328556 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.348332 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.348304 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qlx7"] Jul 22 02:19:58.354584 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.354560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0934c3d1-503f-49e4-8a19-235af2f466fb-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.354725 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.354614 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0934c3d1-503f-49e4-8a19-235af2f466fb-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.354725 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.354668 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x5vx\" (UniqueName: \"kubernetes.io/projected/0934c3d1-503f-49e4-8a19-235af2f466fb-kube-api-access-8x5vx\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.355115 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.355091 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/0934c3d1-503f-49e4-8a19-235af2f466fb-config\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.358655 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.358635 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/0934c3d1-503f-49e4-8a19-235af2f466fb-serving-cert\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.359640 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.359625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.362791 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.362719 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.362903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.362884 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 22 02:19:58.362996 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.362978 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.363422 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.363401 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-jtkwn\"" Jul 22 02:19:58.363611 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.363578 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 22 02:19:58.364914 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.364895 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qlx7"] Jul 22 02:19:58.368848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.368823 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 22 02:19:58.369104 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.369081 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x5vx\" (UniqueName: \"kubernetes.io/projected/0934c3d1-503f-49e4-8a19-235af2f466fb-kube-api-access-8x5vx\") pod \"kube-storage-version-migrator-operator-7658d9b749-bf5jw\" (UID: \"0934c3d1-503f-49e4-8a19-235af2f466fb\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.448092 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.447994 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4"] Jul 22 02:19:58.455487 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455466 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8c6rk\" (UniqueName: \"kubernetes.io/projected/ab3eb0fa-f69a-4497-a301-affe6659886c-kube-api-access-8c6rk\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.455564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455511 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-default-certificate\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.455564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-stats-auth\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.455564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455551 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7790b3f2-3b87-4be0-8355-12fb6275ed23-config\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.455716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455599 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc82755c-0dce-45a5-bd49-a787c95dc17e-serving-cert\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.455716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455644 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.455716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455675 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-trusted-ca\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.455716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455708 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srsr7\" (UniqueName: \"kubernetes.io/projected/31e8d6c2-faae-4b29-b4c6-ccc402a15086-kube-api-access-srsr7\") pod \"network-check-source-7bc5b6bf85-4jw55\" (UID: \"31e8d6c2-faae-4b29-b4c6-ccc402a15086\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" Jul 22 02:19:58.455904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455759 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-config\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.455904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455785 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-serving-cert\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.455904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455812 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6zdks\" (UniqueName: \"kubernetes.io/projected/167c8564-82aa-4e98-8e1f-5314e4c84256-kube-api-access-6zdks\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.455904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455839 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-service-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.455904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7790b3f2-3b87-4be0-8355-12fb6275ed23-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455932 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455956 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-tmp\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.455980 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67gnn\" (UniqueName: \"kubernetes.io/projected/7790b3f2-3b87-4be0-8355-12fb6275ed23-kube-api-access-67gnn\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtx9r\" (UniqueName: \"kubernetes.io/projected/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-kube-api-access-wtx9r\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456070 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/167c8564-82aa-4e98-8e1f-5314e4c84256-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.456118 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456103 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.456338 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456182 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.456338 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-snapshots\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.456338 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.456245 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7pt6\" (UniqueName: \"kubernetes.io/projected/fc82755c-0dce-45a5-bd49-a787c95dc17e-kube-api-access-c7pt6\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.465805 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.465768 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl"] Jul 22 02:19:58.465945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.465827 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" Jul 22 02:19:58.468332 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.468308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" Jul 22 02:19:58.468567 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.468535 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.468567 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.468536 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.468803 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.468788 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-5dqtp\"" Jul 22 02:19:58.480108 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.480085 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4"] Jul 22 02:19:58.480228 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.480124 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl"] Jul 22 02:19:58.480228 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.480215 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.482880 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.482858 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 22 02:19:58.482982 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.482939 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 22 02:19:58.483049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.483006 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:19:58.483188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.483079 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6t7fg\"" Jul 22 02:19:58.557673 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557639 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtx9r\" (UniqueName: \"kubernetes.io/projected/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-kube-api-access-wtx9r\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.557832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557693 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/167c8564-82aa-4e98-8e1f-5314e4c84256-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.557832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.557832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557760 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.557832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557795 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-snapshots\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.557832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557827 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpcxz\" (UniqueName: \"kubernetes.io/projected/395d9965-3ac4-426f-9408-9f9d69424a12-kube-api-access-qpcxz\") pod \"volume-data-source-validator-78cf4c6f6-4nnf4\" (UID: \"395d9965-3ac4-426f-9408-9f9d69424a12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557864 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7pt6\" (UniqueName: \"kubernetes.io/projected/fc82755c-0dce-45a5-bd49-a787c95dc17e-kube-api-access-c7pt6\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557909 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8c6rk\" (UniqueName: \"kubernetes.io/projected/ab3eb0fa-f69a-4497-a301-affe6659886c-kube-api-access-8c6rk\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557951 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-default-certificate\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.557976 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-stats-auth\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.558014 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:19:59.057992389 +0000 UTC m=+40.836068327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:19:58.558081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558044 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7790b3f2-3b87-4be0-8355-12fb6275ed23-config\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558081 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc82755c-0dce-45a5-bd49-a787c95dc17e-serving-cert\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558109 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558158 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-trusted-ca\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558185 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srsr7\" (UniqueName: \"kubernetes.io/projected/31e8d6c2-faae-4b29-b4c6-ccc402a15086-kube-api-access-srsr7\") pod \"network-check-source-7bc5b6bf85-4jw55\" (UID: \"31e8d6c2-faae-4b29-b4c6-ccc402a15086\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-config\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558256 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-serving-cert\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558282 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6zdks\" (UniqueName: \"kubernetes.io/projected/167c8564-82aa-4e98-8e1f-5314e4c84256-kube-api-access-6zdks\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558306 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-service-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558332 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7790b3f2-3b87-4be0-8355-12fb6275ed23-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558390 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.558412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558415 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-tmp\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.558909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558441 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67gnn\" (UniqueName: \"kubernetes.io/projected/7790b3f2-3b87-4be0-8355-12fb6275ed23-kube-api-access-67gnn\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.558909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.558815 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-snapshots\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.559061 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.559039 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-trusted-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.559117 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.559065 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/7790b3f2-3b87-4be0-8355-12fb6275ed23-config\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.559632 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.559604 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:19:58.559761 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.559667 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:19:59.059651937 +0000 UTC m=+40.837727859 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:19:58.560154 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.560110 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-config\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.560587 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.560251 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:19:58.560587 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.560303 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:59.060288709 +0000 UTC m=+40.838364635 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:19:58.560587 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.560337 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/fc82755c-0dce-45a5-bd49-a787c95dc17e-tmp\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.560587 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.560550 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-trusted-ca\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.561503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.561392 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fc82755c-0dce-45a5-bd49-a787c95dc17e-service-ca-bundle\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.561503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.561467 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-serving-cert\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.561765 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.561740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/7790b3f2-3b87-4be0-8355-12fb6275ed23-serving-cert\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.561959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.561937 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-stats-auth\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.562446 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.562425 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-default-certificate\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.562891 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.562869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/167c8564-82aa-4e98-8e1f-5314e4c84256-telemetry-config\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.562958 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.562932 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fc82755c-0dce-45a5-bd49-a787c95dc17e-serving-cert\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.567670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.567509 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtx9r\" (UniqueName: \"kubernetes.io/projected/df65680e-1ec8-47cf-bbcd-c553bf9f81c2-kube-api-access-wtx9r\") pod \"console-operator-565b7b569d-9qlx7\" (UID: \"df65680e-1ec8-47cf-bbcd-c553bf9f81c2\") " pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.567763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.567734 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7pt6\" (UniqueName: \"kubernetes.io/projected/fc82755c-0dce-45a5-bd49-a787c95dc17e-kube-api-access-c7pt6\") pod \"insights-operator-555d9d7885-l4vnz\" (UID: \"fc82755c-0dce-45a5-bd49-a787c95dc17e\") " pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.568109 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.568084 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8c6rk\" (UniqueName: \"kubernetes.io/projected/ab3eb0fa-f69a-4497-a301-affe6659886c-kube-api-access-8c6rk\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:58.568823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.568758 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67gnn\" (UniqueName: \"kubernetes.io/projected/7790b3f2-3b87-4be0-8355-12fb6275ed23-kube-api-access-67gnn\") pod \"service-ca-operator-5bcbc94c78-5c8rc\" (UID: \"7790b3f2-3b87-4be0-8355-12fb6275ed23\") " pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.569038 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.568924 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srsr7\" (UniqueName: \"kubernetes.io/projected/31e8d6c2-faae-4b29-b4c6-ccc402a15086-kube-api-access-srsr7\") pod \"network-check-source-7bc5b6bf85-4jw55\" (UID: \"31e8d6c2-faae-4b29-b4c6-ccc402a15086\") " pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" Jul 22 02:19:58.569543 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.569511 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6zdks\" (UniqueName: \"kubernetes.io/projected/167c8564-82aa-4e98-8e1f-5314e4c84256-kube-api-access-6zdks\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:58.579393 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.579366 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" Jul 22 02:19:58.608225 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.608192 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" Jul 22 02:19:58.620110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.620077 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" Jul 22 02:19:58.626121 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.625958 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw"] Jul 22 02:19:58.628765 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:58.628692 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0934c3d1_503f_49e4_8a19_235af2f466fb.slice/crio-cbf3453d9d5226aa045740867697a57a157396ea0d8ce3687fd56a62f2cbfdf1 WatchSource:0}: Error finding container cbf3453d9d5226aa045740867697a57a157396ea0d8ce3687fd56a62f2cbfdf1: Status 404 returned error can't find the container with id cbf3453d9d5226aa045740867697a57a157396ea0d8ce3687fd56a62f2cbfdf1 Jul 22 02:19:58.667454 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.664441 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.667454 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.664736 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpcxz\" (UniqueName: \"kubernetes.io/projected/395d9965-3ac4-426f-9408-9f9d69424a12-kube-api-access-qpcxz\") pod \"volume-data-source-validator-78cf4c6f6-4nnf4\" (UID: \"395d9965-3ac4-426f-9408-9f9d69424a12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" Jul 22 02:19:58.667454 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.664838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kd7q6\" (UniqueName: \"kubernetes.io/projected/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-kube-api-access-kd7q6\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.678404 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.678196 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:19:58.683905 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.681288 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpcxz\" (UniqueName: \"kubernetes.io/projected/395d9965-3ac4-426f-9408-9f9d69424a12-kube-api-access-qpcxz\") pod \"volume-data-source-validator-78cf4c6f6-4nnf4\" (UID: \"395d9965-3ac4-426f-9408-9f9d69424a12\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" Jul 22 02:19:58.765318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.765242 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55"] Jul 22 02:19:58.766216 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.765607 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kd7q6\" (UniqueName: \"kubernetes.io/projected/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-kube-api-access-kd7q6\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.766216 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.765715 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.766216 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.765911 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:19:58.766216 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.765971 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:19:59.265952102 +0000 UTC m=+41.044028040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:19:58.768515 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:58.768459 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e8d6c2_faae_4b29_b4c6_ccc402a15086.slice/crio-6fa668d96f69a7aa363c93370dd7a4939cc6ec9e96a536f3519e003d51c3a645 WatchSource:0}: Error finding container 6fa668d96f69a7aa363c93370dd7a4939cc6ec9e96a536f3519e003d51c3a645: Status 404 returned error can't find the container with id 6fa668d96f69a7aa363c93370dd7a4939cc6ec9e96a536f3519e003d51c3a645 Jul 22 02:19:58.779629 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.778040 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" Jul 22 02:19:58.779629 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.779300 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kd7q6\" (UniqueName: \"kubernetes.io/projected/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-kube-api-access-kd7q6\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:58.817153 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.816573 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc"] Jul 22 02:19:58.821430 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:58.821398 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7790b3f2_3b87_4be0_8355_12fb6275ed23.slice/crio-116b8b3c3aaea6d9fb8cc79e37b2e25af8628162cb3aabee8b83006fc4d881e6 WatchSource:0}: Error finding container 116b8b3c3aaea6d9fb8cc79e37b2e25af8628162cb3aabee8b83006fc4d881e6: Status 404 returned error can't find the container with id 116b8b3c3aaea6d9fb8cc79e37b2e25af8628162cb3aabee8b83006fc4d881e6 Jul 22 02:19:58.834177 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.834114 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-555d9d7885-l4vnz"] Jul 22 02:19:58.867325 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.867218 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:19:58.867432 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.867323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:19:58.867553 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.867529 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:19:58.867627 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.867577 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:19:58.867671 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.867634 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.86761581 +0000 UTC m=+44.645691732 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:19:58.867671 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.867647 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:19:58.867767 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:58.867703 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.867687441 +0000 UTC m=+44.645763382 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:19:58.868310 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.868290 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-565b7b569d-9qlx7"] Jul 22 02:19:58.871475 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:58.871454 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf65680e_1ec8_47cf_bbcd_c553bf9f81c2.slice/crio-df714f2af59886462d53b0cc289835b74374a202289d8208c95899d74fb676f2 WatchSource:0}: Error finding container df714f2af59886462d53b0cc289835b74374a202289d8208c95899d74fb676f2: Status 404 returned error can't find the container with id df714f2af59886462d53b0cc289835b74374a202289d8208c95899d74fb676f2 Jul 22 02:19:58.934397 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:58.934374 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4"] Jul 22 02:19:58.936339 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:19:58.936317 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod395d9965_3ac4_426f_9408_9f9d69424a12.slice/crio-c6482101aded9575e6e8227dde4126b1171247f5b174c0ebe60714578181214f WatchSource:0}: Error finding container c6482101aded9575e6e8227dde4126b1171247f5b174c0ebe60714578181214f: Status 404 returned error can't find the container with id c6482101aded9575e6e8227dde4126b1171247f5b174c0ebe60714578181214f Jul 22 02:19:59.032735 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.032430 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4jq5p" event={"ID":"bdae54c5-819f-46d8-80e4-0160b8ae9732","Type":"ContainerStarted","Data":"c3f71b3f7f370d51abe11ba19e0fb4fe427b194577198cf67e0e10ef6f950891"} Jul 22 02:19:59.035741 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.035709 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" event={"ID":"31e8d6c2-faae-4b29-b4c6-ccc402a15086","Type":"ContainerStarted","Data":"6fa668d96f69a7aa363c93370dd7a4939cc6ec9e96a536f3519e003d51c3a645"} Jul 22 02:19:59.036864 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.036843 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" event={"ID":"fc82755c-0dce-45a5-bd49-a787c95dc17e","Type":"ContainerStarted","Data":"36ba1514ae6fb6209a6b250b819a34ec3c350e1bc741a8bd106bb5675520d85f"} Jul 22 02:19:59.037846 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.037818 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" event={"ID":"7790b3f2-3b87-4be0-8355-12fb6275ed23","Type":"ContainerStarted","Data":"116b8b3c3aaea6d9fb8cc79e37b2e25af8628162cb3aabee8b83006fc4d881e6"} Jul 22 02:19:59.038898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.038878 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" event={"ID":"df65680e-1ec8-47cf-bbcd-c553bf9f81c2","Type":"ContainerStarted","Data":"df714f2af59886462d53b0cc289835b74374a202289d8208c95899d74fb676f2"} Jul 22 02:19:59.039876 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.039858 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" event={"ID":"0934c3d1-503f-49e4-8a19-235af2f466fb","Type":"ContainerStarted","Data":"cbf3453d9d5226aa045740867697a57a157396ea0d8ce3687fd56a62f2cbfdf1"} Jul 22 02:19:59.040851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.040836 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" event={"ID":"395d9965-3ac4-426f-9408-9f9d69424a12","Type":"ContainerStarted","Data":"c6482101aded9575e6e8227dde4126b1171247f5b174c0ebe60714578181214f"} Jul 22 02:19:59.049244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.049207 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4jq5p" podStartSLOduration=1.265231148 podStartE2EDuration="4.049195088s" totalImagesPullingTime="2.78396394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:19:59.048631417 +0000 UTC m=+40.826707360" watchObservedRunningTime="2026-07-22 02:19:59.049195088 +0000 UTC m=+40.827271028" Jul 22 02:19:59.069873 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.069846 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:59.069960 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.069907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:19:59.070007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.069958 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:19:59.070007 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.069984 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:19:59.070066 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.070041 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:00.070025438 +0000 UTC m=+41.848101356 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:19:59.070066 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.070055 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:19:59.070147 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.070103 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:00.070088752 +0000 UTC m=+41.848164673 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:19:59.070191 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.070150 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:00.070118855 +0000 UTC m=+41.848194775 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:19:59.271870 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:19:59.271833 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:19:59.272788 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.271987 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:19:59.272788 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:19:59.272048 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:00.272031145 +0000 UTC m=+42.050107064 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:20:00.079768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:00.079731 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:00.079925 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:00.079859 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:00.079986 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:00.079947 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:00.080082 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.080065 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:20:00.080154 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.080148 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.08011264 +0000 UTC m=+43.858188561 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:20:00.080575 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.080544 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:00.080654 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.080595 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.080579659 +0000 UTC m=+43.858655580 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:00.081212 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.081195 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.081179377 +0000 UTC m=+43.859255299 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:20:00.281907 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:00.281870 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:00.282375 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.282061 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:20:00.282375 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:00.282122 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:02.282103349 +0000 UTC m=+44.060179273 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:20:02.099055 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.099017 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.099078 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.099127 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.099183 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.099230 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.099245 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.099230335 +0000 UTC m=+47.877306254 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.099259 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.099252948 +0000 UTC m=+47.877328868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:20:02.099442 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.099277 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.099264437 +0000 UTC m=+47.877340355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:02.300576 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.300539 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:02.300768 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.300712 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:20:02.300840 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.300792 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:06.300774547 +0000 UTC m=+48.078850471 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.905596 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:02.905698 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.905712 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.905770 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.90575437 +0000 UTC m=+52.683830289 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.905808 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.905819 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:20:02.905880 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:02.905858 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.905848093 +0000 UTC m=+52.683924012 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:20:06.071800 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.071763 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" event={"ID":"fc82755c-0dce-45a5-bd49-a787c95dc17e","Type":"ContainerStarted","Data":"88b5c9bbb87b53dfb1a4848f541f73fa16805a7b5313fd2b1fa37f8eed798613"} Jul 22 02:20:06.074150 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.074109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" event={"ID":"7790b3f2-3b87-4be0-8355-12fb6275ed23","Type":"ContainerStarted","Data":"771c4d0fc01c9f4fd184a19da7724a852cbaca21f9b654735d34b396438daee9"} Jul 22 02:20:06.075926 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.075903 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/0.log" Jul 22 02:20:06.076017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.075943 2570 generic.go:356] "Generic (PLEG): container finished" podID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" containerID="2bbf7e871a6dd17606ea3650d78b67c0916ce0403413aa4dda0a162967998753" exitCode=255 Jul 22 02:20:06.076017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.075984 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" event={"ID":"df65680e-1ec8-47cf-bbcd-c553bf9f81c2","Type":"ContainerDied","Data":"2bbf7e871a6dd17606ea3650d78b67c0916ce0403413aa4dda0a162967998753"} Jul 22 02:20:06.076308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.076293 2570 scope.go:117] "RemoveContainer" containerID="2bbf7e871a6dd17606ea3650d78b67c0916ce0403413aa4dda0a162967998753" Jul 22 02:20:06.079625 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.079598 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" event={"ID":"0934c3d1-503f-49e4-8a19-235af2f466fb","Type":"ContainerStarted","Data":"27018879277fc56a79d285d38a4804a5668a05f7a13ad150c06165ec7a076d4e"} Jul 22 02:20:06.081285 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.081265 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" event={"ID":"395d9965-3ac4-426f-9408-9f9d69424a12","Type":"ContainerStarted","Data":"77b2a41dfbd141e814421a1aefe2ee31427b5c714e50283ca3037d32cd29ad1f"} Jul 22 02:20:06.083378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.083355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" event={"ID":"31e8d6c2-faae-4b29-b4c6-ccc402a15086","Type":"ContainerStarted","Data":"f2fccfbb0909bd360c4a5d4b28e378fa1b6a6960f898e9bb0374e484e4be7a6f"} Jul 22 02:20:06.090322 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.090276 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" podStartSLOduration=1.253679389 podStartE2EDuration="8.090260503s" totalImagesPullingTime="6.836581114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.089335939 +0000 UTC m=+47.867411882" watchObservedRunningTime="2026-07-22 02:20:06.090260503 +0000 UTC m=+47.868336447" Jul 22 02:20:06.109872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.109307 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7bc5b6bf85-4jw55" podStartSLOduration=1.206692555 podStartE2EDuration="8.109288617s" totalImagesPullingTime="6.902596062s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.108431595 +0000 UTC m=+47.886507538" watchObservedRunningTime="2026-07-22 02:20:06.109288617 +0000 UTC m=+47.887364558" Jul 22 02:20:06.129714 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.128460 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" podStartSLOduration=1.267508037 podStartE2EDuration="8.128438217s" totalImagesPullingTime="6.86093018s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.126504799 +0000 UTC m=+47.904580740" watchObservedRunningTime="2026-07-22 02:20:06.128438217 +0000 UTC m=+47.906514138" Jul 22 02:20:06.134796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.133973 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:06.134796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.134189 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:06.134796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.134315 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:06.135064 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.135034 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:06.135146 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.135093 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.135076344 +0000 UTC m=+55.913152267 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:06.135448 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.135427 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:20:06.135514 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.135481 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.135465152 +0000 UTC m=+55.913541088 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:20:06.136021 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.136001 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.135985929 +0000 UTC m=+55.914061853 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:20:06.168693 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.168118 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" podStartSLOduration=1.121861644 podStartE2EDuration="8.168096522s" totalImagesPullingTime="7.046234878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.166395731 +0000 UTC m=+47.944471673" watchObservedRunningTime="2026-07-22 02:20:06.168096522 +0000 UTC m=+47.946172465" Jul 22 02:20:06.168693 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.168362 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78cf4c6f6-4nnf4" podStartSLOduration=1.428894168 podStartE2EDuration="8.16835246s" totalImagesPullingTime="6.739458292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:06.147350022 +0000 UTC m=+47.925425963" watchObservedRunningTime="2026-07-22 02:20:06.16835246 +0000 UTC m=+47.946428403" Jul 22 02:20:06.337051 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.337007 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:06.337215 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.337197 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:20:06.337283 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:06.337271 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.337250118 +0000 UTC m=+56.115326053 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:20:06.810839 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.810808 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-s2m8q"] Jul 22 02:20:06.837049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.837020 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2m8q"] Jul 22 02:20:06.837229 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.837184 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:06.840155 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.840095 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 22 02:20:06.840155 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.840119 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 22 02:20:06.840410 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.840395 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sgmx\"" Jul 22 02:20:06.942900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.942863 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12307f20-b8c0-433a-86a9-64ad31c18fcd-data-volume\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:06.943088 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.942910 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5xsf\" (UniqueName: \"kubernetes.io/projected/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-api-access-x5xsf\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:06.943088 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.942953 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12307f20-b8c0-433a-86a9-64ad31c18fcd-crio-socket\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:06.943088 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.943009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:06.943244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:06.943199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044498 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044498 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12307f20-b8c0-433a-86a9-64ad31c18fcd-data-volume\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044521 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5xsf\" (UniqueName: \"kubernetes.io/projected/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-api-access-x5xsf\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.044539 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044561 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12307f20-b8c0-433a-86a9-64ad31c18fcd-crio-socket\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.044605 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.544584881 +0000 UTC m=+49.322660813 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : configmap "kube-rbac-proxy" not found Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.044668 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.044730 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.044702 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:07.544691717 +0000 UTC m=+49.322767639 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.045286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044789 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/12307f20-b8c0-433a-86a9-64ad31c18fcd-crio-socket\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.045286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.044869 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/12307f20-b8c0-433a-86a9-64ad31c18fcd-data-volume\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.054794 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.054765 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5xsf\" (UniqueName: \"kubernetes.io/projected/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-api-access-x5xsf\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.088249 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.088222 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/1.log" Jul 22 02:20:07.088665 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.088648 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/0.log" Jul 22 02:20:07.088717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.088688 2570 generic.go:356] "Generic (PLEG): container finished" podID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" exitCode=255 Jul 22 02:20:07.088801 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.088778 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" event={"ID":"df65680e-1ec8-47cf-bbcd-c553bf9f81c2","Type":"ContainerDied","Data":"5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078"} Jul 22 02:20:07.088848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.088835 2570 scope.go:117] "RemoveContainer" containerID="2bbf7e871a6dd17606ea3650d78b67c0916ce0403413aa4dda0a162967998753" Jul 22 02:20:07.089201 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.089034 2570 scope.go:117] "RemoveContainer" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" Jul 22 02:20:07.089294 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.089253 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:07.549113 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.549075 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.549281 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:07.549177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:07.549281 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.549221 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:07.549281 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.549263 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:07.549416 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.549314 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:08.549294727 +0000 UTC m=+50.327370652 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : configmap "kube-rbac-proxy" not found Jul 22 02:20:07.549468 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:07.549425 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:08.549415493 +0000 UTC m=+50.327491415 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:08.096736 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.096709 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/1.log" Jul 22 02:20:08.097169 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.096995 2570 scope.go:117] "RemoveContainer" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" Jul 22 02:20:08.097169 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:08.097126 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:08.214774 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.214743 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cqcd8_61f01f5c-b54c-4b97-8af8-e59860e23e78/dns-node-resolver/0.log" Jul 22 02:20:08.558334 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.558298 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:08.558542 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.558411 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:08.558542 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:08.558473 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:08.558660 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:08.558551 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.558528922 +0000 UTC m=+52.336604841 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:08.558660 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:08.558477 2570 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 22 02:20:08.558660 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:08.558624 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:10.558611289 +0000 UTC m=+52.336687208 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : configmap "kube-rbac-proxy" not found Jul 22 02:20:08.681832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.681794 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:08.681985 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:08.681890 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:09.100280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:09.100252 2570 scope.go:117] "RemoveContainer" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" Jul 22 02:20:09.100662 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:09.100395 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:09.201643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:09.201617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gv24m_708a0732-481b-42fa-af89-ee23dfedff12/node-ca/0.log" Jul 22 02:20:10.001804 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.001774 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4jq5p_bdae54c5-819f-46d8-80e4-0160b8ae9732/serve-healthcheck-canary/0.log" Jul 22 02:20:10.575529 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.575496 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:10.575999 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.575567 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:10.575999 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.575696 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:10.575999 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.575757 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:14.575743439 +0000 UTC m=+56.353819357 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:10.576267 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.576177 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/12307f20-b8c0-433a-86a9-64ad31c18fcd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:10.978453 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.978418 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:10.978613 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:10.978528 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:10.978613 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.978564 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 22 02:20:10.978682 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.978631 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls podName:32b449dc-4ee5-49e3-8543-de214f550403 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:26.978614294 +0000 UTC m=+68.756690212 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls") pod "dns-default-8dr7b" (UID: "32b449dc-4ee5-49e3-8543-de214f550403") : secret "dns-default-metrics-tls" not found Jul 22 02:20:10.978682 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.978653 2570 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 22 02:20:10.978682 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.978667 2570 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5c6497d587-8xlnj: secret "image-registry-tls" not found Jul 22 02:20:10.978780 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:10.978707 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls podName:03c76b22-db24-4449-869f-e2c6745bcbbc nodeName:}" failed. No retries permitted until 2026-07-22 02:20:26.978696198 +0000 UTC m=+68.756772117 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls") pod "image-registry-5c6497d587-8xlnj" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc") : secret "image-registry-tls" not found Jul 22 02:20:14.207359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:14.207323 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:14.207396 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:14.207430 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.207498 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:30.207479883 +0000 UTC m=+71.985555805 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : configmap references non-existent config key: service-ca.crt Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.207528 2570 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.207567 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls podName:167c8564-82aa-4e98-8e1f-5314e4c84256 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:30.20755557 +0000 UTC m=+71.985631489 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-76cf8966f8-qmcb9" (UID: "167c8564-82aa-4e98-8e1f-5314e4c84256") : secret "cluster-monitoring-operator-tls" not found Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.207588 2570 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 22 02:20:14.207953 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.207669 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs podName:ab3eb0fa-f69a-4497-a301-affe6659886c nodeName:}" failed. No retries permitted until 2026-07-22 02:20:30.207651656 +0000 UTC m=+71.985727579 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs") pod "router-default-6d6ff7ddd6-r7kpn" (UID: "ab3eb0fa-f69a-4497-a301-affe6659886c") : secret "router-metrics-certs-default" not found Jul 22 02:20:14.409486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:14.409450 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:14.409641 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.409595 2570 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 22 02:20:14.409681 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.409652 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls podName:3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37 nodeName:}" failed. No retries permitted until 2026-07-22 02:20:30.409637517 +0000 UTC m=+72.187713459 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls") pod "cluster-samples-operator-79b7c55f87-c4vkl" (UID: "3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37") : secret "samples-operator-tls" not found Jul 22 02:20:14.610650 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:14.610562 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:14.610794 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.610699 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 22 02:20:14.610794 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:14.610767 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls podName:12307f20-b8c0-433a-86a9-64ad31c18fcd nodeName:}" failed. No retries permitted until 2026-07-22 02:20:22.61075245 +0000 UTC m=+64.388828369 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls") pod "insights-runtime-extractor-s2m8q" (UID: "12307f20-b8c0-433a-86a9-64ad31c18fcd") : secret "insights-runtime-extractor-tls" not found Jul 22 02:20:20.009045 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:20.009018 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-7wdfd" Jul 22 02:20:22.680367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:22.680326 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:22.682776 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:22.682751 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/12307f20-b8c0-433a-86a9-64ad31c18fcd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-s2m8q\" (UID: \"12307f20-b8c0-433a-86a9-64ad31c18fcd\") " pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:22.751404 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:22.751377 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-6sgmx\"" Jul 22 02:20:22.758781 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:22.758766 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-s2m8q" Jul 22 02:20:22.881473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:22.881443 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-s2m8q"] Jul 22 02:20:22.884239 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:22.884211 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod12307f20_b8c0_433a_86a9_64ad31c18fcd.slice/crio-d32d78e933b53318c3162888d9f33a9bcabe66a2d6888beb9a4e248bbdb96b3f WatchSource:0}: Error finding container d32d78e933b53318c3162888d9f33a9bcabe66a2d6888beb9a4e248bbdb96b3f: Status 404 returned error can't find the container with id d32d78e933b53318c3162888d9f33a9bcabe66a2d6888beb9a4e248bbdb96b3f Jul 22 02:20:23.142302 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:23.142268 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2m8q" event={"ID":"12307f20-b8c0-433a-86a9-64ad31c18fcd","Type":"ContainerStarted","Data":"d32d78e933b53318c3162888d9f33a9bcabe66a2d6888beb9a4e248bbdb96b3f"} Jul 22 02:20:23.743779 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:23.743745 2570 scope.go:117] "RemoveContainer" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" Jul 22 02:20:24.147914 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.147836 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:20:24.148320 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.148294 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/1.log" Jul 22 02:20:24.148440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.148336 2570 generic.go:356] "Generic (PLEG): container finished" podID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" containerID="ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080" exitCode=255 Jul 22 02:20:24.148440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.148402 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" event={"ID":"df65680e-1ec8-47cf-bbcd-c553bf9f81c2","Type":"ContainerDied","Data":"ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080"} Jul 22 02:20:24.148552 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.148461 2570 scope.go:117] "RemoveContainer" containerID="5a064a220137ec1e23ebc03a14ef880895ee7a1684d2d9807c4054f68dfc2078" Jul 22 02:20:24.148929 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.148909 2570 scope.go:117] "RemoveContainer" containerID="ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080" Jul 22 02:20:24.149138 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:24.149110 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:24.598121 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.598090 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:20:24.598323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.598176 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:20:24.601079 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.601057 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 22 02:20:24.601216 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.601057 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 22 02:20:24.611001 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.610977 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/5f68fb46-7000-44a1-af02-354bd9b17c4a-original-pull-secret\") pod \"global-pull-secret-syncer-5bbrj\" (UID: \"5f68fb46-7000-44a1-af02-354bd9b17c4a\") " pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:20:24.611112 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.611003 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f-metrics-certs\") pod \"network-metrics-daemon-944xj\" (UID: \"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f\") " pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:20:24.658249 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.658217 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-5bbrj" Jul 22 02:20:24.674091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.674051 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t8kv7\"" Jul 22 02:20:24.681580 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.681555 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-944xj" Jul 22 02:20:24.699639 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.699616 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:20:24.702410 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.702391 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcfl7\" (UniqueName: \"kubernetes.io/projected/2a0830c2-c123-43d3-a033-b6eabe699142-kube-api-access-xcfl7\") pod \"network-check-target-256x6\" (UID: \"2a0830c2-c123-43d3-a033-b6eabe699142\") " pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:20:24.848911 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.848831 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-5bbrj"] Jul 22 02:20:24.852861 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:24.852813 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f68fb46_7000_44a1_af02_354bd9b17c4a.slice/crio-4b321c3de3752962d80980f01156e9599577d005cd3f97d090ed000b9a49a35b WatchSource:0}: Error finding container 4b321c3de3752962d80980f01156e9599577d005cd3f97d090ed000b9a49a35b: Status 404 returned error can't find the container with id 4b321c3de3752962d80980f01156e9599577d005cd3f97d090ed000b9a49a35b Jul 22 02:20:24.872069 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.872047 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-944xj"] Jul 22 02:20:24.876837 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:24.876490 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4ba9aa5d_a6c6_42f0_b4d4_0a61d8612f2f.slice/crio-0b1f3e09fe50e26acb1d08c51c598cc637f23a443abf6b57f0a825c073209223 WatchSource:0}: Error finding container 0b1f3e09fe50e26acb1d08c51c598cc637f23a443abf6b57f0a825c073209223: Status 404 returned error can't find the container with id 0b1f3e09fe50e26acb1d08c51c598cc637f23a443abf6b57f0a825c073209223 Jul 22 02:20:24.968525 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.968490 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8wzn5\"" Jul 22 02:20:24.976016 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:24.975994 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:20:25.118557 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:25.118486 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-256x6"] Jul 22 02:20:25.152932 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:25.152904 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:20:25.154275 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:25.154248 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-944xj" event={"ID":"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f","Type":"ContainerStarted","Data":"0b1f3e09fe50e26acb1d08c51c598cc637f23a443abf6b57f0a825c073209223"} Jul 22 02:20:25.155414 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:25.155394 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5bbrj" event={"ID":"5f68fb46-7000-44a1-af02-354bd9b17c4a","Type":"ContainerStarted","Data":"4b321c3de3752962d80980f01156e9599577d005cd3f97d090ed000b9a49a35b"} Jul 22 02:20:27.023119 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.023087 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:27.023517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.023230 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:27.025621 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.025591 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/32b449dc-4ee5-49e3-8543-de214f550403-metrics-tls\") pod \"dns-default-8dr7b\" (UID: \"32b449dc-4ee5-49e3-8543-de214f550403\") " pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:27.025730 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.025618 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"image-registry-5c6497d587-8xlnj\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:27.089584 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:27.089539 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a0830c2_c123_43d3_a033_b6eabe699142.slice/crio-3b7f8fcda63a4c37a0437710fc6d375061bf72aeef26481a9cc08e1ecc3d8b1c WatchSource:0}: Error finding container 3b7f8fcda63a4c37a0437710fc6d375061bf72aeef26481a9cc08e1ecc3d8b1c: Status 404 returned error can't find the container with id 3b7f8fcda63a4c37a0437710fc6d375061bf72aeef26481a9cc08e1ecc3d8b1c Jul 22 02:20:27.163611 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.163579 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-256x6" event={"ID":"2a0830c2-c123-43d3-a033-b6eabe699142","Type":"ContainerStarted","Data":"3b7f8fcda63a4c37a0437710fc6d375061bf72aeef26481a9cc08e1ecc3d8b1c"} Jul 22 02:20:27.227564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.227537 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-f4l4l\"" Jul 22 02:20:27.235814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.235792 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:27.247738 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.247711 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-4k4rg\"" Jul 22 02:20:27.255256 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:27.255228 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:28.088778 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.088735 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:20:28.106802 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.106775 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-8dr7b"] Jul 22 02:20:28.170031 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.169899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-256x6" event={"ID":"2a0830c2-c123-43d3-a033-b6eabe699142","Type":"ContainerStarted","Data":"98e231ccd4b21fc8326e29b4d4cbdb073a1194281147432def0a91f3f40a69ba"} Jul 22 02:20:28.170232 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.170172 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:20:28.206778 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.206345 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-256x6" podStartSLOduration=69.206324695 podStartE2EDuration="1m9.206324695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:28.20587421 +0000 UTC m=+69.983950155" watchObservedRunningTime="2026-07-22 02:20:28.206324695 +0000 UTC m=+69.984400647" Jul 22 02:20:28.247505 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:28.247472 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod03c76b22_db24_4449_869f_e2c6745bcbbc.slice/crio-21eeaa7e2755f4ae5424a15e48302914c4b6d6f09215058f04e3d2ec820c1e43 WatchSource:0}: Error finding container 21eeaa7e2755f4ae5424a15e48302914c4b6d6f09215058f04e3d2ec820c1e43: Status 404 returned error can't find the container with id 21eeaa7e2755f4ae5424a15e48302914c4b6d6f09215058f04e3d2ec820c1e43 Jul 22 02:20:28.248240 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:28.248214 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod32b449dc_4ee5_49e3_8543_de214f550403.slice/crio-c08773b1d33168e6731486de23b0c87606865ad25cdfcf4d1b996cc0874c7058 WatchSource:0}: Error finding container c08773b1d33168e6731486de23b0c87606865ad25cdfcf4d1b996cc0874c7058: Status 404 returned error can't find the container with id c08773b1d33168e6731486de23b0c87606865ad25cdfcf4d1b996cc0874c7058 Jul 22 02:20:28.681565 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.681484 2570 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:28.681565 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.681559 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:28.681957 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:28.681938 2570 scope.go:117] "RemoveContainer" containerID="ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080" Jul 22 02:20:28.682189 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:28.682162 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:29.176594 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.176555 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dr7b" event={"ID":"32b449dc-4ee5-49e3-8543-de214f550403","Type":"ContainerStarted","Data":"c08773b1d33168e6731486de23b0c87606865ad25cdfcf4d1b996cc0874c7058"} Jul 22 02:20:29.180499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.180467 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" event={"ID":"03c76b22-db24-4449-869f-e2c6745bcbbc","Type":"ContainerStarted","Data":"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5"} Jul 22 02:20:29.180641 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.180523 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" event={"ID":"03c76b22-db24-4449-869f-e2c6745bcbbc","Type":"ContainerStarted","Data":"21eeaa7e2755f4ae5424a15e48302914c4b6d6f09215058f04e3d2ec820c1e43"} Jul 22 02:20:29.181054 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.181034 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:29.183336 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.183308 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-944xj" event={"ID":"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f","Type":"ContainerStarted","Data":"e13eabe1e576aeba16546a474aef05e9601d9a4c8f5457e9a08fd976baf6b600"} Jul 22 02:20:29.183443 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.183360 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-944xj" event={"ID":"4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f","Type":"ContainerStarted","Data":"d41973fc2b38520f2f516abcabef86f81bf9ad0493165b6ea58bd4122777f828"} Jul 22 02:20:29.187375 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.187309 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2m8q" event={"ID":"12307f20-b8c0-433a-86a9-64ad31c18fcd","Type":"ContainerStarted","Data":"83fabc1a2e6bcfde40d9ce222b89becdc77dd38b3e34f3dd6592b08d39d5d118"} Jul 22 02:20:29.203847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.203786 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" podStartSLOduration=70.203766925 podStartE2EDuration="1m10.203766925s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:29.201385357 +0000 UTC m=+70.979461299" watchObservedRunningTime="2026-07-22 02:20:29.203766925 +0000 UTC m=+70.981842867" Jul 22 02:20:29.219623 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:29.219567 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-944xj" podStartSLOduration=66.772560139 podStartE2EDuration="1m10.219543545s" totalImagesPullingTime="3.446983406s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:29.218183159 +0000 UTC m=+70.996259103" watchObservedRunningTime="2026-07-22 02:20:29.219543545 +0000 UTC m=+70.997619492" Jul 22 02:20:30.257239 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.257196 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:30.257718 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.257291 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:30.257718 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.257350 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:30.258516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.258486 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ab3eb0fa-f69a-4497-a301-affe6659886c-service-ca-bundle\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:30.261674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.261648 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/167c8564-82aa-4e98-8e1f-5314e4c84256-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-76cf8966f8-qmcb9\" (UID: \"167c8564-82aa-4e98-8e1f-5314e4c84256\") " pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:30.262094 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.261940 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/ab3eb0fa-f69a-4497-a301-affe6659886c-metrics-certs\") pod \"router-default-6d6ff7ddd6-r7kpn\" (UID: \"ab3eb0fa-f69a-4497-a301-affe6659886c\") " pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:30.400574 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.400538 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-mw6nm\"" Jul 22 02:20:30.408763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.408738 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" Jul 22 02:20:30.436600 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.436559 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-d655c\"" Jul 22 02:20:30.444652 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.444625 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:30.460054 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.460024 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:30.462593 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.462568 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37-samples-operator-tls\") pod \"cluster-samples-operator-79b7c55f87-c4vkl\" (UID: \"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:30.612907 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.612834 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-6t7fg\"" Jul 22 02:20:30.621188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:30.621158 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" Jul 22 02:20:34.624617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:34.624593 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9"] Jul 22 02:20:34.750359 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:34.750299 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod167c8564_82aa_4e98_8e1f_5314e4c84256.slice/crio-2baad793b1cef4df27ec28088fa58c6c5c46758cc1b274f061562352dd97ed97 WatchSource:0}: Error finding container 2baad793b1cef4df27ec28088fa58c6c5c46758cc1b274f061562352dd97ed97: Status 404 returned error can't find the container with id 2baad793b1cef4df27ec28088fa58c6c5c46758cc1b274f061562352dd97ed97 Jul 22 02:20:34.844351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:34.844321 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl"] Jul 22 02:20:34.847444 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:34.847420 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-6d6ff7ddd6-r7kpn"] Jul 22 02:20:34.850625 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:34.850602 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab3eb0fa_f69a_4497_a301_affe6659886c.slice/crio-9b0977e97616355c503e6d6747f37d6517d2b0f46764f6ca9c22b9247e7cc25a WatchSource:0}: Error finding container 9b0977e97616355c503e6d6747f37d6517d2b0f46764f6ca9c22b9247e7cc25a: Status 404 returned error can't find the container with id 9b0977e97616355c503e6d6747f37d6517d2b0f46764f6ca9c22b9247e7cc25a Jul 22 02:20:35.210739 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.210698 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-5bbrj" event={"ID":"5f68fb46-7000-44a1-af02-354bd9b17c4a","Type":"ContainerStarted","Data":"1e493dc4e661763a32cc9e42446f1e37a8d7f6f6813cb8ece015d5c27e75b730"} Jul 22 02:20:35.211843 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.211822 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" event={"ID":"167c8564-82aa-4e98-8e1f-5314e4c84256","Type":"ContainerStarted","Data":"2baad793b1cef4df27ec28088fa58c6c5c46758cc1b274f061562352dd97ed97"} Jul 22 02:20:35.213253 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.213225 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" event={"ID":"ab3eb0fa-f69a-4497-a301-affe6659886c","Type":"ContainerStarted","Data":"44a477774c04d1e413e882fc533fe5085e2b14df293e195a9ef0fe402f3f9e50"} Jul 22 02:20:35.213359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.213269 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" event={"ID":"ab3eb0fa-f69a-4497-a301-affe6659886c","Type":"ContainerStarted","Data":"9b0977e97616355c503e6d6747f37d6517d2b0f46764f6ca9c22b9247e7cc25a"} Jul 22 02:20:35.214340 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.214320 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" event={"ID":"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37","Type":"ContainerStarted","Data":"65d95818bed32fdb888271926c7ba9a574a15d12dd9413ee1c3507399ca7e71d"} Jul 22 02:20:35.215848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.215826 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2m8q" event={"ID":"12307f20-b8c0-433a-86a9-64ad31c18fcd","Type":"ContainerStarted","Data":"66177d691e2144bca0fe6134f9d383d4404cf385940bc79ef5f6ab16e6806448"} Jul 22 02:20:35.217304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.217286 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dr7b" event={"ID":"32b449dc-4ee5-49e3-8543-de214f550403","Type":"ContainerStarted","Data":"a64e252f4d9187510f55e374d7cfc90d584a6de7b775813ee4835556625957a1"} Jul 22 02:20:35.217376 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.217319 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-8dr7b" event={"ID":"32b449dc-4ee5-49e3-8543-de214f550403","Type":"ContainerStarted","Data":"609bcb9e463f23b35caf53188e226ce0edc4cffb509e55ba8aa497d6eb550ec8"} Jul 22 02:20:35.217423 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.217414 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:35.245694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.245643 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-5bbrj" podStartSLOduration=66.635481842 podStartE2EDuration="1m16.245626572s" totalImagesPullingTime="9.61014473s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:19:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:35.244906609 +0000 UTC m=+77.022982564" watchObservedRunningTime="2026-07-22 02:20:35.245626572 +0000 UTC m=+77.023702512" Jul 22 02:20:35.273193 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.273153 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-8dr7b" podStartSLOduration=34.070943923 podStartE2EDuration="40.273114297s" totalImagesPullingTime="6.202170374s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:35.272366215 +0000 UTC m=+77.050442154" watchObservedRunningTime="2026-07-22 02:20:35.273114297 +0000 UTC m=+77.051190274" Jul 22 02:20:35.307240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.307175 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" podStartSLOduration=37.307155695 podStartE2EDuration="37.307155695s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:35.305210835 +0000 UTC m=+77.083286775" watchObservedRunningTime="2026-07-22 02:20:35.307155695 +0000 UTC m=+77.085231636" Jul 22 02:20:35.314109 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.314083 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl"] Jul 22 02:20:35.330675 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.330651 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.335488 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.335311 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 22 02:20:35.335630 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.335184 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 22 02:20:35.337647 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.336448 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 22 02:20:35.337647 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.336691 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 22 02:20:35.337647 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.336695 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-q6tvb\"" Jul 22 02:20:35.338086 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.338071 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl"] Jul 22 02:20:35.445693 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.445659 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:35.448207 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.448187 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:35.501773 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.501698 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/305603d6-c313-4b45-8ef2-06bc7e0f9055-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.501773 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.501747 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6pctd\" (UniqueName: \"kubernetes.io/projected/305603d6-c313-4b45-8ef2-06bc7e0f9055-kube-api-access-6pctd\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.602466 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.602427 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/305603d6-c313-4b45-8ef2-06bc7e0f9055-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.602637 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.602487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6pctd\" (UniqueName: \"kubernetes.io/projected/305603d6-c313-4b45-8ef2-06bc7e0f9055-kube-api-access-6pctd\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.607072 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.607023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/305603d6-c313-4b45-8ef2-06bc7e0f9055-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.614060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.614027 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6pctd\" (UniqueName: \"kubernetes.io/projected/305603d6-c313-4b45-8ef2-06bc7e0f9055-kube-api-access-6pctd\") pod \"managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl\" (UID: \"305603d6-c313-4b45-8ef2-06bc7e0f9055\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.658162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.658114 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" Jul 22 02:20:35.854466 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:35.854431 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl"] Jul 22 02:20:36.226060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:36.224936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" event={"ID":"305603d6-c313-4b45-8ef2-06bc7e0f9055","Type":"ContainerStarted","Data":"949bdb0fb10d92929b3e023483c0f1befb04c60d60a56506070c1fe5aac8fad0"} Jul 22 02:20:36.226060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:36.225772 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:36.226863 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:36.226840 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-6d6ff7ddd6-r7kpn" Jul 22 02:20:40.241976 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.241426 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-s2m8q" event={"ID":"12307f20-b8c0-433a-86a9-64ad31c18fcd","Type":"ContainerStarted","Data":"f2619907a98d260bee04e3d90d18da39b8e34b97794a95fd49734318bfda0f2f"} Jul 22 02:20:40.244694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.244666 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" event={"ID":"167c8564-82aa-4e98-8e1f-5314e4c84256","Type":"ContainerStarted","Data":"0825eb4ad2304cca63e1dbaea6e4a14eedd6179f9aa3584edc0ca4fc174f976c"} Jul 22 02:20:40.247084 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.247059 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" event={"ID":"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37","Type":"ContainerStarted","Data":"b458840878e42720ec7f5eac46dc151d9284bb76ef22336fe9d6c9f4ea1cf612"} Jul 22 02:20:40.247192 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.247109 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" event={"ID":"3e0b8c14-e0ee-43d1-86a7-9f4eecd18f37","Type":"ContainerStarted","Data":"e695ef3b339d97a9faa9f4131d8f0e7a4b98374275dca937fc2a3c8fffb139af"} Jul 22 02:20:40.263093 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.263038 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-s2m8q" podStartSLOduration=17.910644057 podStartE2EDuration="34.263019789s" totalImagesPullingTime="16.352375732s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:40.262058391 +0000 UTC m=+82.040134332" watchObservedRunningTime="2026-07-22 02:20:40.263019789 +0000 UTC m=+82.041095727" Jul 22 02:20:40.280644 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.280589 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-76cf8966f8-qmcb9" podStartSLOduration=37.6189257 podStartE2EDuration="42.280574722s" totalImagesPullingTime="4.661649022s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:40.280023307 +0000 UTC m=+82.058099247" watchObservedRunningTime="2026-07-22 02:20:40.280574722 +0000 UTC m=+82.058650704" Jul 22 02:20:40.298099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:40.298049 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-79b7c55f87-c4vkl" podStartSLOduration=37.776388725 podStartE2EDuration="42.298033405s" totalImagesPullingTime="4.52164468s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:40.297716602 +0000 UTC m=+82.075792544" watchObservedRunningTime="2026-07-22 02:20:40.298033405 +0000 UTC m=+82.076109344" Jul 22 02:20:42.256145 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:42.256107 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" event={"ID":"305603d6-c313-4b45-8ef2-06bc7e0f9055","Type":"ContainerStarted","Data":"487b558281d6ed02a8b441bdc63cf9f8ebd666cea0fe29507e4e8abfbbfd44af"} Jul 22 02:20:42.274264 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:42.274207 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-5c499b6dd6-hlbhl" podStartSLOduration=1.7477403900000001 podStartE2EDuration="7.274191515s" totalImagesPullingTime="5.526451125s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:20:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:42.272914812 +0000 UTC m=+84.050990751" watchObservedRunningTime="2026-07-22 02:20:42.274191515 +0000 UTC m=+84.052267455" Jul 22 02:20:42.743440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:42.743413 2570 scope.go:117] "RemoveContainer" containerID="ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080" Jul 22 02:20:42.743610 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:20:42.743565 2570 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-565b7b569d-9qlx7_openshift-console-operator(df65680e-1ec8-47cf-bbcd-c553bf9f81c2)\"" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podUID="df65680e-1ec8-47cf-bbcd-c553bf9f81c2" Jul 22 02:20:45.227789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:45.227761 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-8dr7b" Jul 22 02:20:51.199379 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:51.199353 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:20:54.743092 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:54.743059 2570 scope.go:117] "RemoveContainer" containerID="ef34c72d593fea7aa11abd2486a270bf767120657d39b5380b01e603a9d88080" Jul 22 02:20:55.303517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:55.303493 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:20:55.303670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:55.303590 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" event={"ID":"df65680e-1ec8-47cf-bbcd-c553bf9f81c2","Type":"ContainerStarted","Data":"25fe4455f8042b47f1dec7d086cf6389169ca9b9116a0944aa4528455ec3f50c"} Jul 22 02:20:55.303892 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:55.303869 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:55.325394 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:55.325351 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" podStartSLOduration=50.521260343 podStartE2EDuration="57.325337659s" totalImagesPullingTime="6.804077316s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:19:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:55.324325501 +0000 UTC m=+97.102401443" watchObservedRunningTime="2026-07-22 02:20:55.325337659 +0000 UTC m=+97.103413601" Jul 22 02:20:56.000461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.000433 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-565b7b569d-9qlx7" Jul 22 02:20:56.714107 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.714079 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hmt6j"] Jul 22 02:20:56.717310 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.717293 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.720626 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.720598 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 22 02:20:56.721912 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.721897 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 22 02:20:56.721912 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.721904 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-d62vw\"" Jul 22 02:20:56.722057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.721926 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 22 02:20:56.722057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.721933 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 22 02:20:56.755939 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.755907 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc8zx\" (UniqueName: \"kubernetes.io/projected/54d38012-2e57-4c53-a365-974e5d064832-kube-api-access-rc8zx\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.755939 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.755943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-metrics-client-ca\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756160 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-root\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756160 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756078 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-textfile\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756160 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-tls\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756160 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756119 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-wtmp\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756350 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756160 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756350 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.756350 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.756271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-sys\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857346 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-metrics-client-ca\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857561 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-root\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857561 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857429 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-textfile\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857561 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857483 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-root\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857561 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857535 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-tls\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-wtmp\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857600 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857638 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857679 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-sys\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857724 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc8zx\" (UniqueName: \"kubernetes.io/projected/54d38012-2e57-4c53-a365-974e5d064832-kube-api-access-rc8zx\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.857766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857754 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-textfile\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.858046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857789 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-wtmp\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.858046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857796 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54d38012-2e57-4c53-a365-974e5d064832-sys\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.858046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.857987 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-metrics-client-ca\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.858253 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.858230 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-accelerators-collector-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.860222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.860201 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-tls\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.860222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.860212 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/54d38012-2e57-4c53-a365-974e5d064832-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:56.869471 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:56.869448 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc8zx\" (UniqueName: \"kubernetes.io/projected/54d38012-2e57-4c53-a365-974e5d064832-kube-api-access-rc8zx\") pod \"node-exporter-hmt6j\" (UID: \"54d38012-2e57-4c53-a365-974e5d064832\") " pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:57.026349 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:57.026266 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hmt6j" Jul 22 02:20:57.037877 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:20:57.037849 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54d38012_2e57_4c53_a365_974e5d064832.slice/crio-7c054d2cb79211262612b6e94f892433739c31d692e27f101b8289e68864364a WatchSource:0}: Error finding container 7c054d2cb79211262612b6e94f892433739c31d692e27f101b8289e68864364a: Status 404 returned error can't find the container with id 7c054d2cb79211262612b6e94f892433739c31d692e27f101b8289e68864364a Jul 22 02:20:57.051850 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:57.051822 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:20:57.312056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:57.311976 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmt6j" event={"ID":"54d38012-2e57-4c53-a365-974e5d064832","Type":"ContainerStarted","Data":"7c054d2cb79211262612b6e94f892433739c31d692e27f101b8289e68864364a"} Jul 22 02:20:58.317838 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:58.317796 2570 generic.go:356] "Generic (PLEG): container finished" podID="54d38012-2e57-4c53-a365-974e5d064832" containerID="6c8aa8d114702474ec5f25c65472c7a241d70987fbe68c74ddeeafab3f71f1c4" exitCode=0 Jul 22 02:20:58.318253 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:58.317864 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmt6j" event={"ID":"54d38012-2e57-4c53-a365-974e5d064832","Type":"ContainerDied","Data":"6c8aa8d114702474ec5f25c65472c7a241d70987fbe68c74ddeeafab3f71f1c4"} Jul 22 02:20:59.189940 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:59.189910 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-256x6" Jul 22 02:20:59.323754 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:59.323721 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmt6j" event={"ID":"54d38012-2e57-4c53-a365-974e5d064832","Type":"ContainerStarted","Data":"0572f10065b3f7511923619841bea4d85f23e6a905ac7428ce47f4d1a0b205ec"} Jul 22 02:20:59.324194 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:59.323771 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hmt6j" event={"ID":"54d38012-2e57-4c53-a365-974e5d064832","Type":"ContainerStarted","Data":"b0198761a4397116de031fbf55b7aa900911a597208e1b93fe16fd428c1b37ba"} Jul 22 02:20:59.348791 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:20:59.348743 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hmt6j" podStartSLOduration=2.174779411 podStartE2EDuration="3.348728127s" totalImagesPullingTime="1.103428515s" totalInitContainerRuntime="70.520201ms" isStatefulPod=true podCreationTimestamp="2026-07-22 02:20:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:20:59.347539541 +0000 UTC m=+101.125615480" watchObservedRunningTime="2026-07-22 02:20:59.348728127 +0000 UTC m=+101.126804066" Jul 22 02:21:05.889032 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:05.889000 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:06.083300 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.083229 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:06.083470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.083308 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.087365 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.087341 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 22 02:21:06.087521 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.087378 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 22 02:21:06.087521 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.087382 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 22 02:21:06.088078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.088053 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 22 02:21:06.088328 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.088308 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 22 02:21:06.088410 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.088339 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 22 02:21:06.088467 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.088441 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-kk7vq\"" Jul 22 02:21:06.090566 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.089070 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 22 02:21:06.117896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.117868 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.118049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.117917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.118049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.117986 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d4vj\" (UniqueName: \"kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.118049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.118010 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.118181 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.118050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.118181 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.118066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219275 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219343 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219371 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219401 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219447 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.219518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.219483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4d4vj\" (UniqueName: \"kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.220078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.220055 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.220078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.220053 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.220425 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.220114 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.221905 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.221884 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.221997 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.221982 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.229155 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.229117 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d4vj\" (UniqueName: \"kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj\") pod \"console-55b7b4cf59-82c9m\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.396913 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.396879 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:06.549610 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:06.549578 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:06.552739 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:21:06.552712 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3db5fc3_58bc_4194_9cbc_1dc4f3b9d869.slice/crio-48919ea466cee2ed25df5630c04f06f6d5b20ab3fbef993dab8a694883993fbc WatchSource:0}: Error finding container 48919ea466cee2ed25df5630c04f06f6d5b20ab3fbef993dab8a694883993fbc: Status 404 returned error can't find the container with id 48919ea466cee2ed25df5630c04f06f6d5b20ab3fbef993dab8a694883993fbc Jul 22 02:21:07.355972 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:07.355936 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b7b4cf59-82c9m" event={"ID":"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869","Type":"ContainerStarted","Data":"48919ea466cee2ed25df5630c04f06f6d5b20ab3fbef993dab8a694883993fbc"} Jul 22 02:21:10.369420 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:10.369377 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b7b4cf59-82c9m" event={"ID":"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869","Type":"ContainerStarted","Data":"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951"} Jul 22 02:21:10.397023 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:10.396905 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-55b7b4cf59-82c9m" podStartSLOduration=1.805675274 podStartE2EDuration="5.39688921s" totalImagesPullingTime="3.591213936s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:10.394822758 +0000 UTC m=+112.172898697" watchObservedRunningTime="2026-07-22 02:21:10.39688921 +0000 UTC m=+112.174965150" Jul 22 02:21:14.479853 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.479817 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:21:14.484196 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.484174 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.493985 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.493958 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 22 02:21:14.494953 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.494933 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:21:14.582528 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582497 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99jzl\" (UniqueName: \"kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582685 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582539 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582685 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582558 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582685 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582641 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582784 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582689 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582784 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.582784 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.582736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683615 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683588 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99jzl\" (UniqueName: \"kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683627 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683646 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683664 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683692 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.683783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.683716 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.684372 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.684344 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.684898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.684857 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.684996 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.684898 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.685268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.685246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.685510 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.685351 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.690327 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.690304 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.691731 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.691706 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.693317 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.693294 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-99jzl\" (UniqueName: \"kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl\") pod \"console-f877fc698-9rdzn\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.793294 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.793214 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:14.936500 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:14.936469 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:21:14.939346 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:21:14.939322 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd25c1d2_7b7c_44c3_a370_536e924d498b.slice/crio-fde8423e44786dfd2b83390ad01554822027f11051d9b88679f3f10983e118a2 WatchSource:0}: Error finding container fde8423e44786dfd2b83390ad01554822027f11051d9b88679f3f10983e118a2: Status 404 returned error can't find the container with id fde8423e44786dfd2b83390ad01554822027f11051d9b88679f3f10983e118a2 Jul 22 02:21:15.388434 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:15.388398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f877fc698-9rdzn" event={"ID":"fd25c1d2-7b7c-44c3-a370-536e924d498b","Type":"ContainerStarted","Data":"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a"} Jul 22 02:21:15.388630 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:15.388445 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f877fc698-9rdzn" event={"ID":"fd25c1d2-7b7c-44c3-a370-536e924d498b","Type":"ContainerStarted","Data":"fde8423e44786dfd2b83390ad01554822027f11051d9b88679f3f10983e118a2"} Jul 22 02:21:15.410413 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:15.410359 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-f877fc698-9rdzn" podStartSLOduration=1.410344663 podStartE2EDuration="1.410344663s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:21:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:21:15.409404699 +0000 UTC m=+117.187480639" watchObservedRunningTime="2026-07-22 02:21:15.410344663 +0000 UTC m=+117.188420603" Jul 22 02:21:16.397589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:16.397561 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:16.398005 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:16.397630 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:16.402500 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:16.402482 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:17.398911 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:17.398883 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:22.072153 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.072077 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" podUID="03c76b22-db24-4449-869f-e2c6745bcbbc" containerName="registry" containerID="cri-o://6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5" gracePeriod=30 Jul 22 02:21:22.333404 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.333354 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:21:22.413897 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.413858 2570 generic.go:356] "Generic (PLEG): container finished" podID="fc82755c-0dce-45a5-bd49-a787c95dc17e" containerID="88b5c9bbb87b53dfb1a4848f541f73fa16805a7b5313fd2b1fa37f8eed798613" exitCode=0 Jul 22 02:21:22.414064 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.413925 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" event={"ID":"fc82755c-0dce-45a5-bd49-a787c95dc17e","Type":"ContainerDied","Data":"88b5c9bbb87b53dfb1a4848f541f73fa16805a7b5313fd2b1fa37f8eed798613"} Jul 22 02:21:22.414326 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.414306 2570 scope.go:117] "RemoveContainer" containerID="88b5c9bbb87b53dfb1a4848f541f73fa16805a7b5313fd2b1fa37f8eed798613" Jul 22 02:21:22.415215 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.415197 2570 generic.go:356] "Generic (PLEG): container finished" podID="03c76b22-db24-4449-869f-e2c6745bcbbc" containerID="6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5" exitCode=0 Jul 22 02:21:22.415319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.415244 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" Jul 22 02:21:22.415319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.415282 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" event={"ID":"03c76b22-db24-4449-869f-e2c6745bcbbc","Type":"ContainerDied","Data":"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5"} Jul 22 02:21:22.415401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.415331 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5c6497d587-8xlnj" event={"ID":"03c76b22-db24-4449-869f-e2c6745bcbbc","Type":"ContainerDied","Data":"21eeaa7e2755f4ae5424a15e48302914c4b6d6f09215058f04e3d2ec820c1e43"} Jul 22 02:21:22.415401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.415354 2570 scope.go:117] "RemoveContainer" containerID="6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5" Jul 22 02:21:22.423183 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.423168 2570 scope.go:117] "RemoveContainer" containerID="6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5" Jul 22 02:21:22.423462 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:21:22.423443 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5\": container with ID starting with 6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5 not found: ID does not exist" containerID="6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5" Jul 22 02:21:22.423528 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.423469 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5"} err="failed to get container status \"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5\": rpc error: code = NotFound desc = could not find container \"6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5\": container with ID starting with 6284cf49f62e3f65a9737a2a93fc76191bf0832ea9e3761b3ed58ccef3da2bf5 not found: ID does not exist" Jul 22 02:21:22.451224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451300 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zqfjx\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451377 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451377 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451354 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451383 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451410 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451437 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.451622 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.451489 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") pod \"03c76b22-db24-4449-869f-e2c6745bcbbc\" (UID: \"03c76b22-db24-4449-869f-e2c6745bcbbc\") " Jul 22 02:21:22.453681 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.452815 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:22.455007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.453937 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:22.455007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.454240 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx" (OuterVolumeSpecName: "kube-api-access-zqfjx") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "kube-api-access-zqfjx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:22.455007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.454786 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:22.455259 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.455115 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:22.456554 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.456505 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:22.456651 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.456581 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:22.462960 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.462888 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "03c76b22-db24-4449-869f-e2c6745bcbbc" (UID: "03c76b22-db24-4449-869f-e2c6745bcbbc"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:21:22.552916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552880 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-image-registry-private-configuration\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.552916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552913 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/03c76b22-db24-4449-869f-e2c6745bcbbc-installation-pull-secrets\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552926 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-bound-sa-token\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552938 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552948 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/03c76b22-db24-4449-869f-e2c6745bcbbc-ca-trust-extracted\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552959 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zqfjx\" (UniqueName: \"kubernetes.io/projected/03c76b22-db24-4449-869f-e2c6745bcbbc-kube-api-access-zqfjx\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552969 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-registry-certificates\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.553110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.552980 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/03c76b22-db24-4449-869f-e2c6745bcbbc-trusted-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:22.736061 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.736035 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:21:22.739672 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.739647 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5c6497d587-8xlnj"] Jul 22 02:21:22.746890 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:22.746864 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="03c76b22-db24-4449-869f-e2c6745bcbbc" path="/var/lib/kubelet/pods/03c76b22-db24-4449-869f-e2c6745bcbbc/volumes" Jul 22 02:21:23.421028 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:23.420995 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-555d9d7885-l4vnz" event={"ID":"fc82755c-0dce-45a5-bd49-a787c95dc17e","Type":"ContainerStarted","Data":"ac1eafc22fffbe4b4350bef1c6f50fa614371c38b6e19d3bf30099b8fab5688e"} Jul 22 02:21:24.794240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:24.794200 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:24.794743 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:24.794282 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:24.798838 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:24.798820 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:25.433247 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:25.433219 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:21:25.482107 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:25.482082 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:27.439161 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:27.439067 2570 generic.go:356] "Generic (PLEG): container finished" podID="7790b3f2-3b87-4be0-8355-12fb6275ed23" containerID="771c4d0fc01c9f4fd184a19da7724a852cbaca21f9b654735d34b396438daee9" exitCode=0 Jul 22 02:21:27.439543 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:27.439161 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" event={"ID":"7790b3f2-3b87-4be0-8355-12fb6275ed23","Type":"ContainerDied","Data":"771c4d0fc01c9f4fd184a19da7724a852cbaca21f9b654735d34b396438daee9"} Jul 22 02:21:27.439610 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:27.439572 2570 scope.go:117] "RemoveContainer" containerID="771c4d0fc01c9f4fd184a19da7724a852cbaca21f9b654735d34b396438daee9" Jul 22 02:21:28.444196 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:28.444157 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-5bcbc94c78-5c8rc" event={"ID":"7790b3f2-3b87-4be0-8355-12fb6275ed23","Type":"ContainerStarted","Data":"bf098865b9146b1ae06eae181483fc6d4e5954ad4639ad6aa28877f90055dee8"} Jul 22 02:21:38.487283 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:38.487251 2570 generic.go:356] "Generic (PLEG): container finished" podID="0934c3d1-503f-49e4-8a19-235af2f466fb" containerID="27018879277fc56a79d285d38a4804a5668a05f7a13ad150c06165ec7a076d4e" exitCode=0 Jul 22 02:21:38.487764 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:38.487294 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" event={"ID":"0934c3d1-503f-49e4-8a19-235af2f466fb","Type":"ContainerDied","Data":"27018879277fc56a79d285d38a4804a5668a05f7a13ad150c06165ec7a076d4e"} Jul 22 02:21:38.487764 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:38.487580 2570 scope.go:117] "RemoveContainer" containerID="27018879277fc56a79d285d38a4804a5668a05f7a13ad150c06165ec7a076d4e" Jul 22 02:21:39.495904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:39.495862 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-7658d9b749-bf5jw" event={"ID":"0934c3d1-503f-49e4-8a19-235af2f466fb","Type":"ContainerStarted","Data":"54610e59d49f6faeb4b0732ed7e0f8c44d41344db5589ae0bb77d525b6559023"} Jul 22 02:21:50.509941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.509875 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-55b7b4cf59-82c9m" podUID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" containerName="console" containerID="cri-o://21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951" gracePeriod=15 Jul 22 02:21:50.752034 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.752014 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:50.893240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893125 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893202 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893230 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893266 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893314 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893336 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4d4vj\" (UniqueName: \"kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj\") pod \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\" (UID: \"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869\") " Jul 22 02:21:50.893745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893686 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:50.893745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893694 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca" (OuterVolumeSpecName: "service-ca") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:50.893745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.893690 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config" (OuterVolumeSpecName: "console-config") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:21:50.895768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.895738 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:50.895768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.895751 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:21:50.895768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.895758 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj" (OuterVolumeSpecName: "kube-api-access-4d4vj") pod "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" (UID: "d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869"). InnerVolumeSpecName "kube-api-access-4d4vj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:21:50.993910 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993874 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:50.993910 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993905 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4d4vj\" (UniqueName: \"kubernetes.io/projected/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-kube-api-access-4d4vj\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:50.994099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993919 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-oauth-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:50.994099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993932 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:50.994099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993943 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:50.994099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:50.993953 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869-console-oauth-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:21:51.538024 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.537993 2570 generic.go:356] "Generic (PLEG): container finished" podID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" containerID="21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951" exitCode=0 Jul 22 02:21:51.538461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.538037 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-55b7b4cf59-82c9m" Jul 22 02:21:51.538461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.538082 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b7b4cf59-82c9m" event={"ID":"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869","Type":"ContainerDied","Data":"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951"} Jul 22 02:21:51.538461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.538148 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-55b7b4cf59-82c9m" event={"ID":"d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869","Type":"ContainerDied","Data":"48919ea466cee2ed25df5630c04f06f6d5b20ab3fbef993dab8a694883993fbc"} Jul 22 02:21:51.538461 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.538161 2570 scope.go:117] "RemoveContainer" containerID="21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951" Jul 22 02:21:51.546847 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.546827 2570 scope.go:117] "RemoveContainer" containerID="21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951" Jul 22 02:21:51.547119 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:21:51.547101 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951\": container with ID starting with 21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951 not found: ID does not exist" containerID="21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951" Jul 22 02:21:51.547214 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.547125 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951"} err="failed to get container status \"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951\": rpc error: code = NotFound desc = could not find container \"21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951\": container with ID starting with 21b52b00ce3480a49d208170cfb8489d959a1c2cb39198ff33c497d99f00a951 not found: ID does not exist" Jul 22 02:21:51.575033 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.575012 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:51.585702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:51.585683 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-55b7b4cf59-82c9m"] Jul 22 02:21:52.747688 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:21:52.747652 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" path="/var/lib/kubelet/pods/d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869/volumes" Jul 22 02:22:24.632227 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.632198 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633112 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" containerName="console" Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633153 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" containerName="console" Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633201 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="03c76b22-db24-4449-869f-e2c6745bcbbc" containerName="registry" Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633210 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="03c76b22-db24-4449-869f-e2c6745bcbbc" containerName="registry" Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633369 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3db5fc3-58bc-4194-9cbc-1dc4f3b9d869" containerName="console" Jul 22 02:22:24.636151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.633386 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="03c76b22-db24-4449-869f-e2c6745bcbbc" containerName="registry" Jul 22 02:22:24.638165 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.638116 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.651224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.651201 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:22:24.726672 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726642 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726686 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726713 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726744 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt5gc\" (UniqueName: \"kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.726848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.726815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827419 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827388 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt5gc\" (UniqueName: \"kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827419 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827424 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827445 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827466 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827487 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827531 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.827676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.827564 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.828178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.828122 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.828298 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.828256 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.828541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.828520 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.828703 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.828680 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.830187 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.830156 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.830279 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.830212 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.837370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.837352 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt5gc\" (UniqueName: \"kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc\") pod \"console-558d9b7c96-5b95v\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:24.947586 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:24.947510 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:25.082316 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:25.082289 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:22:25.085184 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:22:25.085156 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d8ec224_76e6_461a_8cb5_c4a0c1dfdc1e.slice/crio-f0f03f0031a2d3254a1f4916a58b297bdc87960dd041e3eeaf9b8f309f0e4c6c WatchSource:0}: Error finding container f0f03f0031a2d3254a1f4916a58b297bdc87960dd041e3eeaf9b8f309f0e4c6c: Status 404 returned error can't find the container with id f0f03f0031a2d3254a1f4916a58b297bdc87960dd041e3eeaf9b8f309f0e4c6c Jul 22 02:22:25.666723 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:25.666694 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558d9b7c96-5b95v" event={"ID":"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e","Type":"ContainerStarted","Data":"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e"} Jul 22 02:22:25.667090 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:25.666741 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558d9b7c96-5b95v" event={"ID":"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e","Type":"ContainerStarted","Data":"f0f03f0031a2d3254a1f4916a58b297bdc87960dd041e3eeaf9b8f309f0e4c6c"} Jul 22 02:22:25.685196 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:25.685112 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-558d9b7c96-5b95v" podStartSLOduration=1.685096053 podStartE2EDuration="1.685096053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:22:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:22:25.683684605 +0000 UTC m=+187.461760556" watchObservedRunningTime="2026-07-22 02:22:25.685096053 +0000 UTC m=+187.463171992" Jul 22 02:22:34.948299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:34.948261 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:34.948747 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:34.948334 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:34.953007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:34.952988 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:35.704339 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:35.704302 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:22:35.754079 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:22:35.754043 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:23:00.777389 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:00.777318 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-f877fc698-9rdzn" podUID="fd25c1d2-7b7c-44c3-a370-536e924d498b" containerName="console" containerID="cri-o://59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a" gracePeriod=15 Jul 22 02:23:01.029646 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.029585 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:23:01.106781 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106747 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.106781 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106783 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106808 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106835 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106863 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-99jzl\" (UniqueName: \"kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106910 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.106939 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config\") pod \"fd25c1d2-7b7c-44c3-a370-536e924d498b\" (UID: \"fd25c1d2-7b7c-44c3-a370-536e924d498b\") " Jul 22 02:23:01.107403 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.107349 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:01.107403 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.107385 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:01.107550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.107447 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config" (OuterVolumeSpecName: "console-config") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:01.107550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.107498 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:23:01.109162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.109112 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:23:01.109264 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.109199 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:23:01.109306 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.109276 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl" (OuterVolumeSpecName: "kube-api-access-99jzl") pod "fd25c1d2-7b7c-44c3-a370-536e924d498b" (UID: "fd25c1d2-7b7c-44c3-a370-536e924d498b"). InnerVolumeSpecName "kube-api-access-99jzl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:23:01.207832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207787 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-trusted-ca-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.207832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207817 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-oauth-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.207832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207841 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-oauth-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.208056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207850 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.208056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207858 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-99jzl\" (UniqueName: \"kubernetes.io/projected/fd25c1d2-7b7c-44c3-a370-536e924d498b-kube-api-access-99jzl\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.208056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207865 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.208056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.207873 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd25c1d2-7b7c-44c3-a370-536e924d498b-console-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:23:01.802768 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.802730 2570 generic.go:356] "Generic (PLEG): container finished" podID="fd25c1d2-7b7c-44c3-a370-536e924d498b" containerID="59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a" exitCode=0 Jul 22 02:23:01.803244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.802785 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-f877fc698-9rdzn" Jul 22 02:23:01.803244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.802800 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f877fc698-9rdzn" event={"ID":"fd25c1d2-7b7c-44c3-a370-536e924d498b","Type":"ContainerDied","Data":"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a"} Jul 22 02:23:01.803244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.802841 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-f877fc698-9rdzn" event={"ID":"fd25c1d2-7b7c-44c3-a370-536e924d498b","Type":"ContainerDied","Data":"fde8423e44786dfd2b83390ad01554822027f11051d9b88679f3f10983e118a2"} Jul 22 02:23:01.803244 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.802860 2570 scope.go:117] "RemoveContainer" containerID="59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a" Jul 22 02:23:01.812860 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.812842 2570 scope.go:117] "RemoveContainer" containerID="59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a" Jul 22 02:23:01.813376 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:23:01.813351 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a\": container with ID starting with 59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a not found: ID does not exist" containerID="59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a" Jul 22 02:23:01.813473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.813384 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a"} err="failed to get container status \"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a\": rpc error: code = NotFound desc = could not find container \"59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a\": container with ID starting with 59511fbb31310dc906e3795f87454b23e0654682f8c4b0f2c7917971b191ef1a not found: ID does not exist" Jul 22 02:23:01.832666 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.832639 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:23:01.838477 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:01.838454 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-f877fc698-9rdzn"] Jul 22 02:23:02.747444 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:02.747414 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd25c1d2-7b7c-44c3-a370-536e924d498b" path="/var/lib/kubelet/pods/fd25c1d2-7b7c-44c3-a370-536e924d498b/volumes" Jul 22 02:23:54.358099 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.358014 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:23:54.358694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.358412 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd25c1d2-7b7c-44c3-a370-536e924d498b" containerName="console" Jul 22 02:23:54.358694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.358426 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd25c1d2-7b7c-44c3-a370-536e924d498b" containerName="console" Jul 22 02:23:54.358694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.358482 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd25c1d2-7b7c-44c3-a370-536e924d498b" containerName="console" Jul 22 02:23:54.361029 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.361007 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.373357 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.373326 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:23:54.402558 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402525 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402568 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvqnh\" (UniqueName: \"kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402616 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402656 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402819 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402819 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402757 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.402819 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.402783 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.503900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.503863 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lvqnh\" (UniqueName: \"kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.503900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.503905 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.503932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.503988 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504026 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504054 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504091 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.504903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504876 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.505018 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504876 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.505018 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504883 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.505018 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.504975 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.506564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.506540 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.506652 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.506590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.512516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.512467 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvqnh\" (UniqueName: \"kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh\") pod \"console-7dddb65df9-bz766\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.671694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.671608 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:23:54.806215 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:54.806077 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:23:54.808908 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:23:54.808882 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1e8bd1_2f4b_4c7b_9315_865ea1dcfe3b.slice/crio-9276d78614607893af16d470e6f29ff51d99692cc77c0a8f0fa4695343bd434c WatchSource:0}: Error finding container 9276d78614607893af16d470e6f29ff51d99692cc77c0a8f0fa4695343bd434c: Status 404 returned error can't find the container with id 9276d78614607893af16d470e6f29ff51d99692cc77c0a8f0fa4695343bd434c Jul 22 02:23:55.000860 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:55.000825 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dddb65df9-bz766" event={"ID":"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b","Type":"ContainerStarted","Data":"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2"} Jul 22 02:23:55.001039 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:55.000886 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dddb65df9-bz766" event={"ID":"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b","Type":"ContainerStarted","Data":"9276d78614607893af16d470e6f29ff51d99692cc77c0a8f0fa4695343bd434c"} Jul 22 02:23:55.020000 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:23:55.019948 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7dddb65df9-bz766" podStartSLOduration=1.019933967 podStartE2EDuration="1.019933967s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:23:54 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:23:55.019190494 +0000 UTC m=+276.797266436" watchObservedRunningTime="2026-07-22 02:23:55.019933967 +0000 UTC m=+276.798009905" Jul 22 02:24:04.671948 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:04.671912 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:24:04.672358 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:04.672000 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:24:04.676805 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:04.676783 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:24:05.044940 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:05.044914 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:24:05.092671 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:05.092642 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:24:18.638702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:18.638673 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:24:18.644242 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:18.644216 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:24:18.647682 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:18.647661 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:24:18.658925 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:18.658905 2570 kubelet.go:1648] "Image garbage collection succeeded" Jul 22 02:24:18.662354 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:18.662335 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:24:30.115327 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.115286 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-558d9b7c96-5b95v" podUID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" containerName="console" containerID="cri-o://3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e" gracePeriod=15 Jul 22 02:24:30.353081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.353055 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:24:30.477186 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477150 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477195 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477236 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477252 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477278 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt5gc\" (UniqueName: \"kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477311 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477367 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477357 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert\") pod \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\" (UID: \"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e\") " Jul 22 02:24:30.477744 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477664 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:24:30.477877 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477733 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config" (OuterVolumeSpecName: "console-config") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:24:30.477877 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.477815 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca" (OuterVolumeSpecName: "service-ca") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:24:30.478262 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.478046 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:24:30.479583 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.479557 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:24:30.479677 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.479589 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:24:30.479722 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.479687 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc" (OuterVolumeSpecName: "kube-api-access-pt5gc") pod "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" (UID: "8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e"). InnerVolumeSpecName "kube-api-access-pt5gc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:24:30.578497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578456 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-oauth-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578488 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578497 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt5gc\" (UniqueName: \"kubernetes.io/projected/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-kube-api-access-pt5gc\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578507 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578760 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578515 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-oauth-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578760 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578523 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-trusted-ca-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:30.578760 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:30.578531 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e-console-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:31.138697 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.138604 2570 generic.go:356] "Generic (PLEG): container finished" podID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" containerID="3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e" exitCode=0 Jul 22 02:24:31.138697 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.138653 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558d9b7c96-5b95v" event={"ID":"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e","Type":"ContainerDied","Data":"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e"} Jul 22 02:24:31.138697 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.138670 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-558d9b7c96-5b95v" Jul 22 02:24:31.139272 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.138739 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-558d9b7c96-5b95v" event={"ID":"8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e","Type":"ContainerDied","Data":"f0f03f0031a2d3254a1f4916a58b297bdc87960dd041e3eeaf9b8f309f0e4c6c"} Jul 22 02:24:31.139272 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.138767 2570 scope.go:117] "RemoveContainer" containerID="3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e" Jul 22 02:24:31.146780 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.146759 2570 scope.go:117] "RemoveContainer" containerID="3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e" Jul 22 02:24:31.147055 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:24:31.147033 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e\": container with ID starting with 3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e not found: ID does not exist" containerID="3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e" Jul 22 02:24:31.147126 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.147062 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e"} err="failed to get container status \"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e\": rpc error: code = NotFound desc = could not find container \"3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e\": container with ID starting with 3d93c00efe52642ee784799d97ad8b8ec4217380a22ea671a59c405224b4db6e not found: ID does not exist" Jul 22 02:24:31.157776 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.157751 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:24:31.163605 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:31.163581 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-558d9b7c96-5b95v"] Jul 22 02:24:32.746828 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:32.746787 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" path="/var/lib/kubelet/pods/8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e/volumes" Jul 22 02:24:40.948172 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.948097 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk"] Jul 22 02:24:40.948613 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.948507 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" containerName="console" Jul 22 02:24:40.948613 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.948522 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" containerName="console" Jul 22 02:24:40.948613 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.948598 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8d8ec224-76e6-461a-8cb5-c4a0c1dfdc1e" containerName="console" Jul 22 02:24:40.951588 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.951569 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:40.954493 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.954477 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 02:24:40.954817 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.954789 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 02:24:40.955718 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.955704 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8gwg9\"" Jul 22 02:24:40.959582 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:40.959559 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk"] Jul 22 02:24:41.059731 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.059701 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.059731 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.059740 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4c2dg\" (UniqueName: \"kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.059957 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.059769 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.160438 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.160405 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.160438 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.160444 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4c2dg\" (UniqueName: \"kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.160619 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.160475 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.160794 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.160763 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.160832 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.160816 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.169490 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.169469 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4c2dg\" (UniqueName: \"kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.260085 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.260057 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:41.390569 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.390521 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk"] Jul 22 02:24:41.392857 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:24:41.392835 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb554939_9709_4faa_90b1_f1cb0b6f7c2e.slice/crio-b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b WatchSource:0}: Error finding container b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b: Status 404 returned error can't find the container with id b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b Jul 22 02:24:41.395049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:41.395035 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:24:42.182896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:42.182855 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" event={"ID":"fb554939-9709-4faa-90b1-f1cb0b6f7c2e","Type":"ContainerStarted","Data":"b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b"} Jul 22 02:24:48.209458 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:48.209421 2570 generic.go:356] "Generic (PLEG): container finished" podID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerID="8ad94871bc3465517d4b737750c4daaa7ab09c4fd9eb4abc961a3ce168606d94" exitCode=0 Jul 22 02:24:48.209806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:48.209465 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" event={"ID":"fb554939-9709-4faa-90b1-f1cb0b6f7c2e","Type":"ContainerDied","Data":"8ad94871bc3465517d4b737750c4daaa7ab09c4fd9eb4abc961a3ce168606d94"} Jul 22 02:24:51.223082 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:51.223049 2570 generic.go:356] "Generic (PLEG): container finished" podID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerID="bfb8a36e5e3a6c3b1754c5d137bdbe3cc0829d314dc111a634bc449d675ea830" exitCode=0 Jul 22 02:24:51.223478 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:51.223155 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" event={"ID":"fb554939-9709-4faa-90b1-f1cb0b6f7c2e","Type":"ContainerDied","Data":"bfb8a36e5e3a6c3b1754c5d137bdbe3cc0829d314dc111a634bc449d675ea830"} Jul 22 02:24:58.252872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:58.252836 2570 generic.go:356] "Generic (PLEG): container finished" podID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerID="fce379e1e8b71cd6a393f3d2dcfb7b30cb9cad2d1432d17293205923f9542cfb" exitCode=0 Jul 22 02:24:58.253238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:58.252899 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" event={"ID":"fb554939-9709-4faa-90b1-f1cb0b6f7c2e","Type":"ContainerDied","Data":"fce379e1e8b71cd6a393f3d2dcfb7b30cb9cad2d1432d17293205923f9542cfb"} Jul 22 02:24:59.382758 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.382733 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:24:59.511492 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.511402 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util\") pod \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " Jul 22 02:24:59.511492 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.511454 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4c2dg\" (UniqueName: \"kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg\") pod \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " Jul 22 02:24:59.511716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.511499 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle\") pod \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\" (UID: \"fb554939-9709-4faa-90b1-f1cb0b6f7c2e\") " Jul 22 02:24:59.512143 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.512108 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle" (OuterVolumeSpecName: "bundle") pod "fb554939-9709-4faa-90b1-f1cb0b6f7c2e" (UID: "fb554939-9709-4faa-90b1-f1cb0b6f7c2e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:24:59.513767 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.513738 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg" (OuterVolumeSpecName: "kube-api-access-4c2dg") pod "fb554939-9709-4faa-90b1-f1cb0b6f7c2e" (UID: "fb554939-9709-4faa-90b1-f1cb0b6f7c2e"). InnerVolumeSpecName "kube-api-access-4c2dg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:24:59.515901 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.515879 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util" (OuterVolumeSpecName: "util") pod "fb554939-9709-4faa-90b1-f1cb0b6f7c2e" (UID: "fb554939-9709-4faa-90b1-f1cb0b6f7c2e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:24:59.612299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.612263 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:59.612299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.612291 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4c2dg\" (UniqueName: \"kubernetes.io/projected/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-kube-api-access-4c2dg\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:24:59.612299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:24:59.612302 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fb554939-9709-4faa-90b1-f1cb0b6f7c2e-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:00.262440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:00.262417 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" Jul 22 02:25:00.262604 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:00.262416 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e675408bvsk" event={"ID":"fb554939-9709-4faa-90b1-f1cb0b6f7c2e","Type":"ContainerDied","Data":"b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b"} Jul 22 02:25:00.262604 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:00.262531 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b24966b4a94d9334d16d684f5ed6765e35f4a0b87144661d85b6c4eab429e81b" Jul 22 02:25:04.105150 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105109 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v"] Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105492 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="util" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105508 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="util" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105519 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="pull" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105527 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="pull" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105535 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="extract" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105543 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="extract" Jul 22 02:25:04.105706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.105621 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="fb554939-9709-4faa-90b1-f1cb0b6f7c2e" containerName="extract" Jul 22 02:25:04.149927 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.149897 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v"] Jul 22 02:25:04.150094 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.149991 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.153238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.153205 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:25:04.153350 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.153324 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 22 02:25:04.153629 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.153612 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-2vxdt\"" Jul 22 02:25:04.247152 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.247105 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg9gz\" (UniqueName: \"kubernetes.io/projected/478f1ab2-27b5-4f0a-9785-9a547b16237e-kube-api-access-xg9gz\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.247311 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.247167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478f1ab2-27b5-4f0a-9785-9a547b16237e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.348218 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.348191 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478f1ab2-27b5-4f0a-9785-9a547b16237e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.348381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.348260 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg9gz\" (UniqueName: \"kubernetes.io/projected/478f1ab2-27b5-4f0a-9785-9a547b16237e-kube-api-access-xg9gz\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.348581 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.348562 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/478f1ab2-27b5-4f0a-9785-9a547b16237e-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.356978 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.356923 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg9gz\" (UniqueName: \"kubernetes.io/projected/478f1ab2-27b5-4f0a-9785-9a547b16237e-kube-api-access-xg9gz\") pod \"cert-manager-operator-controller-manager-55c444bf86-7zn6v\" (UID: \"478f1ab2-27b5-4f0a-9785-9a547b16237e\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.458811 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.458777 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" Jul 22 02:25:04.617864 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:04.617777 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v"] Jul 22 02:25:04.621958 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:04.621933 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod478f1ab2_27b5_4f0a_9785_9a547b16237e.slice/crio-0361b94c8b503cb094e3f6bdad089b746e7b036420310cbe4ea09c4aa16c7e44 WatchSource:0}: Error finding container 0361b94c8b503cb094e3f6bdad089b746e7b036420310cbe4ea09c4aa16c7e44: Status 404 returned error can't find the container with id 0361b94c8b503cb094e3f6bdad089b746e7b036420310cbe4ea09c4aa16c7e44 Jul 22 02:25:05.280598 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:05.280565 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" event={"ID":"478f1ab2-27b5-4f0a-9785-9a547b16237e","Type":"ContainerStarted","Data":"0361b94c8b503cb094e3f6bdad089b746e7b036420310cbe4ea09c4aa16c7e44"} Jul 22 02:25:09.298442 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:09.298353 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" event={"ID":"478f1ab2-27b5-4f0a-9785-9a547b16237e","Type":"ContainerStarted","Data":"754a76745d3811753bc630a46aebfe41ce8f23788d92315e4efab4c2dd38cc2e"} Jul 22 02:25:09.321282 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:09.321231 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-7zn6v" podStartSLOduration=0.943597853 podStartE2EDuration="5.321216748s" totalImagesPullingTime="4.377618895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:09.319005042 +0000 UTC m=+351.097080990" watchObservedRunningTime="2026-07-22 02:25:09.321216748 +0000 UTC m=+351.099292687" Jul 22 02:25:10.297733 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.297656 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pmwm7"] Jul 22 02:25:10.300027 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.300009 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.303321 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.303296 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 22 02:25:10.304460 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.304444 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-6xjw5\"" Jul 22 02:25:10.304547 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.304466 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 22 02:25:10.311656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.311637 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pmwm7"] Jul 22 02:25:10.392203 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.392163 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.392383 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.392241 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4shb\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-kube-api-access-v4shb\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.492823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.492789 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4shb\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-kube-api-access-v4shb\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.492977 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.492852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.503081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.503053 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4shb\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-kube-api-access-v4shb\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.511771 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.511743 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-pmwm7\" (UID: \"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9\") " pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.608513 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.608435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:10.737246 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:10.737222 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-pmwm7"] Jul 22 02:25:10.739790 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:10.739766 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcc5d73fb_b8b1_48a6_bbc4_a8c586d0b7e9.slice/crio-9f54d17d7d15c88d470839a6f9ce3390fb57ff021aa21e09caa641b7c353c6d7 WatchSource:0}: Error finding container 9f54d17d7d15c88d470839a6f9ce3390fb57ff021aa21e09caa641b7c353c6d7: Status 404 returned error can't find the container with id 9f54d17d7d15c88d470839a6f9ce3390fb57ff021aa21e09caa641b7c353c6d7 Jul 22 02:25:11.211693 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.211656 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq"] Jul 22 02:25:11.215355 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.215328 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.218172 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.218153 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 02:25:11.219243 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.219220 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8gwg9\"" Jul 22 02:25:11.219354 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.219228 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 02:25:11.224020 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.223999 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq"] Jul 22 02:25:11.298153 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.298102 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xfbvb\" (UniqueName: \"kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.298330 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.298178 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.298330 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.298206 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.308411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.308383 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" event={"ID":"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9","Type":"ContainerStarted","Data":"9f54d17d7d15c88d470839a6f9ce3390fb57ff021aa21e09caa641b7c353c6d7"} Jul 22 02:25:11.403806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.399278 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.403806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.399382 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.403806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.399468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xfbvb\" (UniqueName: \"kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.403806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.399679 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.403806 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.399952 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.408921 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.408898 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xfbvb\" (UniqueName: \"kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.525252 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.525172 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:11.671825 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:11.671798 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq"] Jul 22 02:25:11.674038 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:11.674014 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaae0c840_30b8_4d7a_840f_7506f70e1be8.slice/crio-e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff WatchSource:0}: Error finding container e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff: Status 404 returned error can't find the container with id e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff Jul 22 02:25:12.314053 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:12.314010 2570 generic.go:356] "Generic (PLEG): container finished" podID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerID="eef55feae35b83207988afd9ffa31a0de22574eee9a28d1d88e8b0c9aabc6ee7" exitCode=0 Jul 22 02:25:12.314537 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:12.314068 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" event={"ID":"aae0c840-30b8-4d7a-840f-7506f70e1be8","Type":"ContainerDied","Data":"eef55feae35b83207988afd9ffa31a0de22574eee9a28d1d88e8b0c9aabc6ee7"} Jul 22 02:25:12.314537 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:12.314112 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" event={"ID":"aae0c840-30b8-4d7a-840f-7506f70e1be8","Type":"ContainerStarted","Data":"e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff"} Jul 22 02:25:14.324299 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:14.324259 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" event={"ID":"cc5d73fb-b8b1-48a6-bbc4-a8c586d0b7e9","Type":"ContainerStarted","Data":"edc014e80acf078c42193a6147e6e096c00bb8768a4dbe51a2dd1ad1c233548f"} Jul 22 02:25:14.324686 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:14.324333 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:14.344564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:14.344511 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" podStartSLOduration=1.328216882 podStartE2EDuration="4.344491041s" totalImagesPullingTime="3.016274159s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:14.343353434 +0000 UTC m=+356.121429375" watchObservedRunningTime="2026-07-22 02:25:14.344491041 +0000 UTC m=+356.122566984" Jul 22 02:25:16.334314 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:16.334283 2570 generic.go:356] "Generic (PLEG): container finished" podID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerID="57bee6d55681108c7bd1f8a2327c482cb4260aa2687832cb1dfd8ec7cd45314c" exitCode=0 Jul 22 02:25:16.334822 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:16.334332 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" event={"ID":"aae0c840-30b8-4d7a-840f-7506f70e1be8","Type":"ContainerDied","Data":"57bee6d55681108c7bd1f8a2327c482cb4260aa2687832cb1dfd8ec7cd45314c"} Jul 22 02:25:17.340931 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:17.340898 2570 generic.go:356] "Generic (PLEG): container finished" podID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerID="2577cfb2abdd9360b98885f4b8db703e6f4446b803b124d04befb4856226bfa8" exitCode=0 Jul 22 02:25:17.341359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:17.340964 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" event={"ID":"aae0c840-30b8-4d7a-840f-7506f70e1be8","Type":"ContainerDied","Data":"2577cfb2abdd9360b98885f4b8db703e6f4446b803b124d04befb4856226bfa8"} Jul 22 02:25:18.466410 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.466385 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:18.557233 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.557199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xfbvb\" (UniqueName: \"kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb\") pod \"aae0c840-30b8-4d7a-840f-7506f70e1be8\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " Jul 22 02:25:18.557407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.557286 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util\") pod \"aae0c840-30b8-4d7a-840f-7506f70e1be8\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " Jul 22 02:25:18.557407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.557323 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle\") pod \"aae0c840-30b8-4d7a-840f-7506f70e1be8\" (UID: \"aae0c840-30b8-4d7a-840f-7506f70e1be8\") " Jul 22 02:25:18.557755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.557727 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle" (OuterVolumeSpecName: "bundle") pod "aae0c840-30b8-4d7a-840f-7506f70e1be8" (UID: "aae0c840-30b8-4d7a-840f-7506f70e1be8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:25:18.559526 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.559494 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb" (OuterVolumeSpecName: "kube-api-access-xfbvb") pod "aae0c840-30b8-4d7a-840f-7506f70e1be8" (UID: "aae0c840-30b8-4d7a-840f-7506f70e1be8"). InnerVolumeSpecName "kube-api-access-xfbvb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:25:18.562256 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.562218 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util" (OuterVolumeSpecName: "util") pod "aae0c840-30b8-4d7a-840f-7506f70e1be8" (UID: "aae0c840-30b8-4d7a-840f-7506f70e1be8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:25:18.657893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.657826 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:18.657893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.657848 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/aae0c840-30b8-4d7a-840f-7506f70e1be8-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:18.657893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:18.657857 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xfbvb\" (UniqueName: \"kubernetes.io/projected/aae0c840-30b8-4d7a-840f-7506f70e1be8-kube-api-access-xfbvb\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:19.355720 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:19.355695 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" Jul 22 02:25:19.355901 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:19.355691 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fzhhcq" event={"ID":"aae0c840-30b8-4d7a-840f-7506f70e1be8","Type":"ContainerDied","Data":"e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff"} Jul 22 02:25:19.355901 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:19.355838 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e45530a023b5ca4ea20aaf02ee9c7114ba353dfa4c116a0f36fe280bd53676ff" Jul 22 02:25:20.331238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:20.331205 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-pmwm7" Jul 22 02:25:32.297580 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297544 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp"] Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297844 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="extract" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297856 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="extract" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297871 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="util" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297877 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="util" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297887 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="pull" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297892 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="pull" Jul 22 02:25:32.297959 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.297940 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="aae0c840-30b8-4d7a-840f-7506f70e1be8" containerName="extract" Jul 22 02:25:32.300418 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.300402 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.303005 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.302982 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 22 02:25:32.303116 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.303069 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8gwg9\"" Jul 22 02:25:32.304015 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.303993 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 22 02:25:32.309006 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.308976 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp"] Jul 22 02:25:32.365573 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.365538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.365573 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.365586 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44zn8\" (UniqueName: \"kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.365801 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.365608 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.466513 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.466473 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.466690 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.466529 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44zn8\" (UniqueName: \"kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.466690 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.466553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.466822 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.466805 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.466872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.466846 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.480043 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.480018 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44zn8\" (UniqueName: \"kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.609471 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.609372 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:32.747320 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:32.747291 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3408b63e_65d6_463f_85ed_6c6cd180b8b2.slice/crio-10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9 WatchSource:0}: Error finding container 10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9: Status 404 returned error can't find the container with id 10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9 Jul 22 02:25:32.748284 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:32.748264 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp"] Jul 22 02:25:33.417936 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:33.417901 2570 generic.go:356] "Generic (PLEG): container finished" podID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerID="3ef3874a209c6f43db8d7e9983cf7339813dbe3312f79f68a7a867380ea4785d" exitCode=0 Jul 22 02:25:33.418458 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:33.417986 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" event={"ID":"3408b63e-65d6-463f-85ed-6c6cd180b8b2","Type":"ContainerDied","Data":"3ef3874a209c6f43db8d7e9983cf7339813dbe3312f79f68a7a867380ea4785d"} Jul 22 02:25:33.418458 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:33.418034 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" event={"ID":"3408b63e-65d6-463f-85ed-6c6cd180b8b2","Type":"ContainerStarted","Data":"10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9"} Jul 22 02:25:34.424472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:34.424435 2570 generic.go:356] "Generic (PLEG): container finished" podID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerID="5aaa4220673017914ca5aae35da7262dd7294d91e5f1db323f31a7d27d1dc6e8" exitCode=0 Jul 22 02:25:34.424472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:34.424471 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" event={"ID":"3408b63e-65d6-463f-85ed-6c6cd180b8b2","Type":"ContainerDied","Data":"5aaa4220673017914ca5aae35da7262dd7294d91e5f1db323f31a7d27d1dc6e8"} Jul 22 02:25:35.431185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:35.431125 2570 generic.go:356] "Generic (PLEG): container finished" podID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerID="c4ee2c66b29925e6b4e45bf348c39e56e1f6f97a6e6164efdf65efc7e7102a30" exitCode=0 Jul 22 02:25:35.431535 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:35.431204 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" event={"ID":"3408b63e-65d6-463f-85ed-6c6cd180b8b2","Type":"ContainerDied","Data":"c4ee2c66b29925e6b4e45bf348c39e56e1f6f97a6e6164efdf65efc7e7102a30"} Jul 22 02:25:36.558182 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.558158 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:36.600060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.600028 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle\") pod \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " Jul 22 02:25:36.600224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.600119 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util\") pod \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " Jul 22 02:25:36.600224 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.600165 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44zn8\" (UniqueName: \"kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8\") pod \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\" (UID: \"3408b63e-65d6-463f-85ed-6c6cd180b8b2\") " Jul 22 02:25:36.600727 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.600701 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle" (OuterVolumeSpecName: "bundle") pod "3408b63e-65d6-463f-85ed-6c6cd180b8b2" (UID: "3408b63e-65d6-463f-85ed-6c6cd180b8b2"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:25:36.602245 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.602225 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8" (OuterVolumeSpecName: "kube-api-access-44zn8") pod "3408b63e-65d6-463f-85ed-6c6cd180b8b2" (UID: "3408b63e-65d6-463f-85ed-6c6cd180b8b2"). InnerVolumeSpecName "kube-api-access-44zn8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:25:36.605712 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.605674 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util" (OuterVolumeSpecName: "util") pod "3408b63e-65d6-463f-85ed-6c6cd180b8b2" (UID: "3408b63e-65d6-463f-85ed-6c6cd180b8b2"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:25:36.701433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.701352 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44zn8\" (UniqueName: \"kubernetes.io/projected/3408b63e-65d6-463f-85ed-6c6cd180b8b2-kube-api-access-44zn8\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:36.701433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.701380 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:36.701433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:36.701389 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3408b63e-65d6-463f-85ed-6c6cd180b8b2-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:25:37.441779 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:37.441752 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" Jul 22 02:25:37.441779 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:37.441768 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afr89tp" event={"ID":"3408b63e-65d6-463f-85ed-6c6cd180b8b2","Type":"ContainerDied","Data":"10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9"} Jul 22 02:25:37.441977 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:37.441814 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10319bd60b508cb33de914fdd73c200a036f82710e0ea257a590817e13035ba9" Jul 22 02:25:41.082950 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.082921 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-2t2wm"] Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083216 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="extract" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083228 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="extract" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083242 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="pull" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083249 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="pull" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083269 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="util" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083276 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="util" Jul 22 02:25:41.083351 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.083322 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3408b63e-65d6-463f-85ed-6c6cd180b8b2" containerName="extract" Jul 22 02:25:41.086029 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.086013 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.089786 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.089760 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 22 02:25:41.090219 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.090198 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 22 02:25:41.091218 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.091195 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 22 02:25:41.091526 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.091510 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-f42xq\"" Jul 22 02:25:41.091746 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.091731 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 22 02:25:41.091823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.091797 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 22 02:25:41.102771 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.102748 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-2t2wm"] Jul 22 02:25:41.132304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.132275 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5jjv\" (UniqueName: \"kubernetes.io/projected/569d85ac-02cd-4800-b584-ffb1b5e0a460-kube-api-access-s5jjv\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.132433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.132306 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-metrics-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.132433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.132343 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.132497 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.132438 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/569d85ac-02cd-4800-b584-ffb1b5e0a460-manager-config\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.233595 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.233547 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.233595 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.233609 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/569d85ac-02cd-4800-b584-ffb1b5e0a460-manager-config\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.233859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.233639 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5jjv\" (UniqueName: \"kubernetes.io/projected/569d85ac-02cd-4800-b584-ffb1b5e0a460-kube-api-access-s5jjv\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.233859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.233661 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-metrics-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.234412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.234293 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/569d85ac-02cd-4800-b584-ffb1b5e0a460-manager-config\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.236194 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.236170 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.236194 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.236191 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/569d85ac-02cd-4800-b584-ffb1b5e0a460-metrics-cert\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.243491 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.243471 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5jjv\" (UniqueName: \"kubernetes.io/projected/569d85ac-02cd-4800-b584-ffb1b5e0a460-kube-api-access-s5jjv\") pod \"lws-controller-manager-645b87676-2t2wm\" (UID: \"569d85ac-02cd-4800-b584-ffb1b5e0a460\") " pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.394452 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.394362 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:41.558303 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:41.558274 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-645b87676-2t2wm"] Jul 22 02:25:41.561416 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:41.561393 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod569d85ac_02cd_4800_b584_ffb1b5e0a460.slice/crio-07afd79aa57fecb61ac12b9471228643514641f66f849ee46ab826e3e331b581 WatchSource:0}: Error finding container 07afd79aa57fecb61ac12b9471228643514641f66f849ee46ab826e3e331b581: Status 404 returned error can't find the container with id 07afd79aa57fecb61ac12b9471228643514641f66f849ee46ab826e3e331b581 Jul 22 02:25:42.468654 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:42.468620 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" event={"ID":"569d85ac-02cd-4800-b584-ffb1b5e0a460","Type":"ContainerStarted","Data":"07afd79aa57fecb61ac12b9471228643514641f66f849ee46ab826e3e331b581"} Jul 22 02:25:44.478687 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:44.478601 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" event={"ID":"569d85ac-02cd-4800-b584-ffb1b5e0a460","Type":"ContainerStarted","Data":"0ac1bce8be85738393d09ddb490726407cc44ebf2e22385056d62af3abd69a6e"} Jul 22 02:25:44.479062 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:44.478740 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:25:44.498008 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:44.497961 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" podStartSLOduration=0.853948346 podStartE2EDuration="3.497945224s" totalImagesPullingTime="2.643996878s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:44.496702362 +0000 UTC m=+386.274778301" watchObservedRunningTime="2026-07-22 02:25:44.497945224 +0000 UTC m=+386.276021163" Jul 22 02:25:45.696998 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.696966 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45"] Jul 22 02:25:45.729291 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.729262 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45"] Jul 22 02:25:45.729440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.729399 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.732352 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.732319 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 22 02:25:45.732352 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.732345 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 22 02:25:45.732549 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.732404 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 22 02:25:45.732612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.732552 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-gf5lf\"" Jul 22 02:25:45.732612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.732604 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 22 02:25:45.771745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.771714 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.771900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.771756 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.771900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.771774 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtlv6\" (UniqueName: \"kubernetes.io/projected/ecddbba7-0af7-45c5-891d-5a2909871ced-kube-api-access-vtlv6\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.771900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.771849 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.772012 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.771901 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.872681 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.872643 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.872859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.872695 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.872859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.872725 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtlv6\" (UniqueName: \"kubernetes.io/projected/ecddbba7-0af7-45c5-891d-5a2909871ced-kube-api-access-vtlv6\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.872859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.872743 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.872859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.872762 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.873165 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.873120 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-manifests\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.875166 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.875119 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ecddbba7-0af7-45c5-891d-5a2909871ced-tmp\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.875264 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.875216 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.875264 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.875248 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ecddbba7-0af7-45c5-891d-5a2909871ced-webhook-cert\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:45.880857 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:45.880839 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtlv6\" (UniqueName: \"kubernetes.io/projected/ecddbba7-0af7-45c5-891d-5a2909871ced-kube-api-access-vtlv6\") pod \"opendatahub-operator-controller-manager-7cb9fd785c-lqx45\" (UID: \"ecddbba7-0af7-45c5-891d-5a2909871ced\") " pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:46.040790 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:46.040763 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:46.217658 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:46.215822 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45"] Jul 22 02:25:46.219532 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:25:46.219508 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecddbba7_0af7_45c5_891d_5a2909871ced.slice/crio-02cf97d9c7ea58f139860fccdca8ad5e8807d27ebe034b14d278186deb1ab6fb WatchSource:0}: Error finding container 02cf97d9c7ea58f139860fccdca8ad5e8807d27ebe034b14d278186deb1ab6fb: Status 404 returned error can't find the container with id 02cf97d9c7ea58f139860fccdca8ad5e8807d27ebe034b14d278186deb1ab6fb Jul 22 02:25:46.488390 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:46.488347 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" event={"ID":"ecddbba7-0af7-45c5-891d-5a2909871ced","Type":"ContainerStarted","Data":"02cf97d9c7ea58f139860fccdca8ad5e8807d27ebe034b14d278186deb1ab6fb"} Jul 22 02:25:49.504467 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:49.504432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" event={"ID":"ecddbba7-0af7-45c5-891d-5a2909871ced","Type":"ContainerStarted","Data":"dbbae0cc656617e50346538d6254b4d89f4b43d3d75d96863a5147004cf9878d"} Jul 22 02:25:50.509796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:50.509763 2570 generic.go:356] "Generic (PLEG): container finished" podID="ecddbba7-0af7-45c5-891d-5a2909871ced" containerID="dbbae0cc656617e50346538d6254b4d89f4b43d3d75d96863a5147004cf9878d" exitCode=0 Jul 22 02:25:50.510216 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:50.509848 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" event={"ID":"ecddbba7-0af7-45c5-891d-5a2909871ced","Type":"ContainerDied","Data":"dbbae0cc656617e50346538d6254b4d89f4b43d3d75d96863a5147004cf9878d"} Jul 22 02:25:51.516911 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:51.516867 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" event={"ID":"ecddbba7-0af7-45c5-891d-5a2909871ced","Type":"ContainerStarted","Data":"6c9c3e58b325172b4bbccf980253ba12edcd5889b69b3858ba6ad0337b2cff63"} Jul 22 02:25:51.517381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:51.516966 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:25:51.543916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:51.543863 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" podStartSLOduration=2.289254732 podStartE2EDuration="6.543848323s" totalImagesPullingTime="3.247586392s" totalInitContainerRuntime="1.007007199s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:25:45 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:25:51.541765528 +0000 UTC m=+393.319841468" watchObservedRunningTime="2026-07-22 02:25:51.543848323 +0000 UTC m=+393.321924262" Jul 22 02:25:55.485674 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:25:55.485644 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-645b87676-2t2wm" Jul 22 02:26:02.523533 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:02.523497 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7cb9fd785c-lqx45" Jul 22 02:26:13.183096 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.183059 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r"] Jul 22 02:26:13.186368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.186344 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.190514 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.190492 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 22 02:26:13.190723 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.190709 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 22 02:26:13.190821 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.190739 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 22 02:26:13.190891 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.190828 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 22 02:26:13.191114 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.191099 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-m7w55\"" Jul 22 02:26:13.207665 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.207640 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r"] Jul 22 02:26:13.303855 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.303814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc4e5add-200a-4613-8202-8a06e4bbbfca-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.303864 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gmfb6\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-kube-api-access-gmfb6\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.303919 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.303961 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.304005 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.304028 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.304298 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.304073 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404486 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404455 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404493 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc4e5add-200a-4613-8202-8a06e4bbbfca-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404522 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gmfb6\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-kube-api-access-gmfb6\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404556 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404577 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404603 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.404691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.404618 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.405188 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.405127 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.406981 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.406949 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/cc4e5add-200a-4613-8202-8a06e4bbbfca-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.407219 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.407203 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.407288 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.407235 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.407450 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.407431 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/cc4e5add-200a-4613-8202-8a06e4bbbfca-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.418310 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.418288 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.418403 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.418367 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gmfb6\" (UniqueName: \"kubernetes.io/projected/cc4e5add-200a-4613-8202-8a06e4bbbfca-kube-api-access-gmfb6\") pod \"istiod-openshift-gateway-94bb8fbfd-gkf5r\" (UID: \"cc4e5add-200a-4613-8202-8a06e4bbbfca\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.495408 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.495379 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:13.635766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:13.635736 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r"] Jul 22 02:26:13.636800 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:26:13.636770 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc4e5add_200a_4613_8202_8a06e4bbbfca.slice/crio-f749be9b49ec04363586dac64d6d8f4d605eb02b8e3f00e1272e96eee70cd12a WatchSource:0}: Error finding container f749be9b49ec04363586dac64d6d8f4d605eb02b8e3f00e1272e96eee70cd12a: Status 404 returned error can't find the container with id f749be9b49ec04363586dac64d6d8f4d605eb02b8e3f00e1272e96eee70cd12a Jul 22 02:26:14.619725 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:14.619689 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" event={"ID":"cc4e5add-200a-4613-8202-8a06e4bbbfca","Type":"ContainerStarted","Data":"f749be9b49ec04363586dac64d6d8f4d605eb02b8e3f00e1272e96eee70cd12a"} Jul 22 02:26:17.852872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:17.852837 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:17.853165 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:17.852905 2570 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 22 02:26:18.641125 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:18.641096 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" event={"ID":"cc4e5add-200a-4613-8202-8a06e4bbbfca","Type":"ContainerStarted","Data":"e181aa1b85d610d040ed2edc469fd0741951db5ba0320e8213c8902442e20ff8"} Jul 22 02:26:18.641423 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:18.641405 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:18.642632 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:18.642607 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" Jul 22 02:26:18.669982 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:26:18.669931 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-gkf5r" podStartSLOduration=1.456116327 podStartE2EDuration="5.669910659s" totalImagesPullingTime="4.213794332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:26:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:26:18.667745789 +0000 UTC m=+420.445821729" watchObservedRunningTime="2026-07-22 02:26:18.669910659 +0000 UTC m=+420.447986600" Jul 22 02:27:21.629807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.629774 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm"] Jul 22 02:27:21.632966 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.632949 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.635840 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.635814 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 02:27:21.636013 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.635980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ss4lb\"" Jul 22 02:27:21.636096 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.636035 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 02:27:21.643394 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.643374 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm"] Jul 22 02:27:21.669374 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.669347 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6rqlb\" (UniqueName: \"kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.669517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.669392 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.669517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.669426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.770174 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.770125 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.770363 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.770211 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6rqlb\" (UniqueName: \"kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.770363 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.770240 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.770564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.770543 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.770564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.770555 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.779782 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.779751 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6rqlb\" (UniqueName: \"kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:21.942470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:21.942395 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:22.070481 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.070457 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm"] Jul 22 02:27:22.073014 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:27:22.072980 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ed129b2_0873_4730_8076_e421f83e89a1.slice/crio-5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f WatchSource:0}: Error finding container 5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f: Status 404 returned error can't find the container with id 5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f Jul 22 02:27:22.220799 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.220766 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth"] Jul 22 02:27:22.223886 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.223870 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.232668 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.232649 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth"] Jul 22 02:27:22.274711 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.274685 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mggzq\" (UniqueName: \"kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.274851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.274731 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.274851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.274805 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.376123 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.376092 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.376297 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.376178 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.376297 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.376215 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mggzq\" (UniqueName: \"kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.376549 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.376525 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.376587 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.376533 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.385044 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.385024 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mggzq\" (UniqueName: \"kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.532831 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.532753 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:22.660286 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.660254 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth"] Jul 22 02:27:22.663021 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:27:22.662999 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc99df528_8927_4321_989e_13bcfbe76134.slice/crio-f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338 WatchSource:0}: Error finding container f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338: Status 404 returned error can't find the container with id f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338 Jul 22 02:27:22.822268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.822196 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv"] Jul 22 02:27:22.825436 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.825421 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.835175 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.835148 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv"] Jul 22 02:27:22.880886 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.880855 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.880886 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.880888 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.881082 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.880908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vw5tt\" (UniqueName: \"kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.917304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.917272 2570 generic.go:356] "Generic (PLEG): container finished" podID="c99df528-8927-4321-989e-13bcfbe76134" containerID="ea7b37701264fff45a9b741c9ac847a79d9396a49b7ed70c52c95d89699d3b5a" exitCode=0 Jul 22 02:27:22.917456 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.917355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" event={"ID":"c99df528-8927-4321-989e-13bcfbe76134","Type":"ContainerDied","Data":"ea7b37701264fff45a9b741c9ac847a79d9396a49b7ed70c52c95d89699d3b5a"} Jul 22 02:27:22.917456 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.917428 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" event={"ID":"c99df528-8927-4321-989e-13bcfbe76134","Type":"ContainerStarted","Data":"f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338"} Jul 22 02:27:22.918998 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.918975 2570 generic.go:356] "Generic (PLEG): container finished" podID="6ed129b2-0873-4730-8076-e421f83e89a1" containerID="cf7ed8fcbfa4ab44834bd8c86ed5e2a032c845e5f60f894ff21deb32f2788644" exitCode=0 Jul 22 02:27:22.919075 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.919017 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" event={"ID":"6ed129b2-0873-4730-8076-e421f83e89a1","Type":"ContainerDied","Data":"cf7ed8fcbfa4ab44834bd8c86ed5e2a032c845e5f60f894ff21deb32f2788644"} Jul 22 02:27:22.919075 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.919051 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" event={"ID":"6ed129b2-0873-4730-8076-e421f83e89a1","Type":"ContainerStarted","Data":"5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f"} Jul 22 02:27:22.981262 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.981234 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.981422 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.981273 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.981422 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.981400 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vw5tt\" (UniqueName: \"kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.981647 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.981630 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.981744 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.981721 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:22.990670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:22.990645 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vw5tt\" (UniqueName: \"kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:23.139123 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.139045 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:23.227235 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.227191 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr"] Jul 22 02:27:23.231098 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.231078 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.245089 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.245064 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr"] Jul 22 02:27:23.278097 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.278049 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv"] Jul 22 02:27:23.280179 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:27:23.280154 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a33f8c1_112a_4035_898b_dab1d04d57fd.slice/crio-b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf WatchSource:0}: Error finding container b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf: Status 404 returned error can't find the container with id b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf Jul 22 02:27:23.283876 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.283852 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.283974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.283893 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5rnq\" (UniqueName: \"kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.283974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.283926 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.384962 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.384932 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.385091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.385022 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.385091 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.385071 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5rnq\" (UniqueName: \"kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.385325 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.385306 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.385401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.385338 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.394627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.394575 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5rnq\" (UniqueName: \"kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.541200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.541171 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:23.827598 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.827573 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr"] Jul 22 02:27:23.851984 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:27:23.851955 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b89c749_870a_40fb_a56d_95f238ac322f.slice/crio-af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b WatchSource:0}: Error finding container af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b: Status 404 returned error can't find the container with id af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b Jul 22 02:27:23.924725 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.924694 2570 generic.go:356] "Generic (PLEG): container finished" podID="c99df528-8927-4321-989e-13bcfbe76134" containerID="fa31afb2bb66bd84556be061cb5813088d540495c53cb801d9e93d88b856f180" exitCode=0 Jul 22 02:27:23.924913 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.924764 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" event={"ID":"c99df528-8927-4321-989e-13bcfbe76134","Type":"ContainerDied","Data":"fa31afb2bb66bd84556be061cb5813088d540495c53cb801d9e93d88b856f180"} Jul 22 02:27:23.925962 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.925943 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" event={"ID":"5b89c749-870a-40fb-a56d-95f238ac322f","Type":"ContainerStarted","Data":"af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b"} Jul 22 02:27:23.927449 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.927430 2570 generic.go:356] "Generic (PLEG): container finished" podID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerID="47d459d2b5783b251fdaee834f2afe5de16bc194fbe0e274fa499bf1d6af518f" exitCode=0 Jul 22 02:27:23.927547 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.927512 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" event={"ID":"0a33f8c1-112a-4035-898b-dab1d04d57fd","Type":"ContainerDied","Data":"47d459d2b5783b251fdaee834f2afe5de16bc194fbe0e274fa499bf1d6af518f"} Jul 22 02:27:23.927547 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.927538 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" event={"ID":"0a33f8c1-112a-4035-898b-dab1d04d57fd","Type":"ContainerStarted","Data":"b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf"} Jul 22 02:27:23.929298 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.929280 2570 generic.go:356] "Generic (PLEG): container finished" podID="6ed129b2-0873-4730-8076-e421f83e89a1" containerID="04bb7b15f74850faef8b3f8199ff9dc1ba7c8b986fa3e9ea65061aed1c0f182e" exitCode=0 Jul 22 02:27:23.929395 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:23.929310 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" event={"ID":"6ed129b2-0873-4730-8076-e421f83e89a1","Type":"ContainerDied","Data":"04bb7b15f74850faef8b3f8199ff9dc1ba7c8b986fa3e9ea65061aed1c0f182e"} Jul 22 02:27:24.935893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.935859 2570 generic.go:356] "Generic (PLEG): container finished" podID="6ed129b2-0873-4730-8076-e421f83e89a1" containerID="97bf1d499439f9a5c9ae0ec8c20e396351906cb9380f32d4ae2e43d09b6ea551" exitCode=0 Jul 22 02:27:24.936315 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.935947 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" event={"ID":"6ed129b2-0873-4730-8076-e421f83e89a1","Type":"ContainerDied","Data":"97bf1d499439f9a5c9ae0ec8c20e396351906cb9380f32d4ae2e43d09b6ea551"} Jul 22 02:27:24.937924 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.937902 2570 generic.go:356] "Generic (PLEG): container finished" podID="c99df528-8927-4321-989e-13bcfbe76134" containerID="59dc05ed4ab58cdc92068e4b98c39df568b3b31e7739266118cb69ff458c1696" exitCode=0 Jul 22 02:27:24.938032 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.937964 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" event={"ID":"c99df528-8927-4321-989e-13bcfbe76134","Type":"ContainerDied","Data":"59dc05ed4ab58cdc92068e4b98c39df568b3b31e7739266118cb69ff458c1696"} Jul 22 02:27:24.939353 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.939336 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b89c749-870a-40fb-a56d-95f238ac322f" containerID="6aa3d587905846892a2f59aea45dd7d00f75a4e1f5ed04c66dee4e862abeb540" exitCode=0 Jul 22 02:27:24.939445 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.939432 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" event={"ID":"5b89c749-870a-40fb-a56d-95f238ac322f","Type":"ContainerDied","Data":"6aa3d587905846892a2f59aea45dd7d00f75a4e1f5ed04c66dee4e862abeb540"} Jul 22 02:27:24.941289 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.941270 2570 generic.go:356] "Generic (PLEG): container finished" podID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerID="a9e68639fbda6c18d423ad423626a147e0bdf440783ba8faf307e27138bc34d0" exitCode=0 Jul 22 02:27:24.941381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:24.941314 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" event={"ID":"0a33f8c1-112a-4035-898b-dab1d04d57fd","Type":"ContainerDied","Data":"a9e68639fbda6c18d423ad423626a147e0bdf440783ba8faf307e27138bc34d0"} Jul 22 02:27:25.948380 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:25.948284 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b89c749-870a-40fb-a56d-95f238ac322f" containerID="870d94134de0d61fe4ad07b49d02fec895e92ba1a822a8fe5dccba66a77b859d" exitCode=0 Jul 22 02:27:25.948809 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:25.948376 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" event={"ID":"5b89c749-870a-40fb-a56d-95f238ac322f","Type":"ContainerDied","Data":"870d94134de0d61fe4ad07b49d02fec895e92ba1a822a8fe5dccba66a77b859d"} Jul 22 02:27:25.950483 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:25.950457 2570 generic.go:356] "Generic (PLEG): container finished" podID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerID="cc201335ce3d8a6e43bc96b6cc7e5797f2e6543bcab6189d6768ed532f51042f" exitCode=0 Jul 22 02:27:25.950566 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:25.950539 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" event={"ID":"0a33f8c1-112a-4035-898b-dab1d04d57fd","Type":"ContainerDied","Data":"cc201335ce3d8a6e43bc96b6cc7e5797f2e6543bcab6189d6768ed532f51042f"} Jul 22 02:27:26.088762 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.088733 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:26.107766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.107600 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle\") pod \"6ed129b2-0873-4730-8076-e421f83e89a1\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " Jul 22 02:27:26.107766 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.107718 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6rqlb\" (UniqueName: \"kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb\") pod \"6ed129b2-0873-4730-8076-e421f83e89a1\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " Jul 22 02:27:26.107971 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.107774 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util\") pod \"6ed129b2-0873-4730-8076-e421f83e89a1\" (UID: \"6ed129b2-0873-4730-8076-e421f83e89a1\") " Jul 22 02:27:26.108904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.108745 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle" (OuterVolumeSpecName: "bundle") pod "6ed129b2-0873-4730-8076-e421f83e89a1" (UID: "6ed129b2-0873-4730-8076-e421f83e89a1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:26.110804 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.110755 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb" (OuterVolumeSpecName: "kube-api-access-6rqlb") pod "6ed129b2-0873-4730-8076-e421f83e89a1" (UID: "6ed129b2-0873-4730-8076-e421f83e89a1"). InnerVolumeSpecName "kube-api-access-6rqlb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:27:26.115896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.115869 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util" (OuterVolumeSpecName: "util") pod "6ed129b2-0873-4730-8076-e421f83e89a1" (UID: "6ed129b2-0873-4730-8076-e421f83e89a1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:26.117108 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.117073 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:26.209124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209098 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mggzq\" (UniqueName: \"kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq\") pod \"c99df528-8927-4321-989e-13bcfbe76134\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " Jul 22 02:27:26.209304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209162 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle\") pod \"c99df528-8927-4321-989e-13bcfbe76134\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " Jul 22 02:27:26.209304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209195 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util\") pod \"c99df528-8927-4321-989e-13bcfbe76134\" (UID: \"c99df528-8927-4321-989e-13bcfbe76134\") " Jul 22 02:27:26.209393 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209341 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6rqlb\" (UniqueName: \"kubernetes.io/projected/6ed129b2-0873-4730-8076-e421f83e89a1-kube-api-access-6rqlb\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.209393 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209359 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.209393 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209373 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/6ed129b2-0873-4730-8076-e421f83e89a1-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.209622 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.209595 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle" (OuterVolumeSpecName: "bundle") pod "c99df528-8927-4321-989e-13bcfbe76134" (UID: "c99df528-8927-4321-989e-13bcfbe76134"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:26.211443 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.211422 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq" (OuterVolumeSpecName: "kube-api-access-mggzq") pod "c99df528-8927-4321-989e-13bcfbe76134" (UID: "c99df528-8927-4321-989e-13bcfbe76134"). InnerVolumeSpecName "kube-api-access-mggzq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:27:26.214122 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.214092 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util" (OuterVolumeSpecName: "util") pod "c99df528-8927-4321-989e-13bcfbe76134" (UID: "c99df528-8927-4321-989e-13bcfbe76134"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:26.309926 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.309883 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mggzq\" (UniqueName: \"kubernetes.io/projected/c99df528-8927-4321-989e-13bcfbe76134-kube-api-access-mggzq\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.309926 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.309915 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.309926 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.309926 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c99df528-8927-4321-989e-13bcfbe76134-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:26.956614 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.956582 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" event={"ID":"c99df528-8927-4321-989e-13bcfbe76134","Type":"ContainerDied","Data":"f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338"} Jul 22 02:27:26.956614 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.956592 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth" Jul 22 02:27:26.957075 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.956628 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f875c16b0f71372c8e3f4a220b59013f5b6538fc0b7ee0a051468b88d9d85338" Jul 22 02:27:26.958736 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.958706 2570 generic.go:356] "Generic (PLEG): container finished" podID="5b89c749-870a-40fb-a56d-95f238ac322f" containerID="5c17f76e275f5ab7c65961ffa587a4f67e9b0d0ed8dba8fa7e230b511f6ffbd7" exitCode=0 Jul 22 02:27:26.958850 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.958814 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" event={"ID":"5b89c749-870a-40fb-a56d-95f238ac322f","Type":"ContainerDied","Data":"5c17f76e275f5ab7c65961ffa587a4f67e9b0d0ed8dba8fa7e230b511f6ffbd7"} Jul 22 02:27:26.960475 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.960460 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" Jul 22 02:27:26.960564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.960488 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm" event={"ID":"6ed129b2-0873-4730-8076-e421f83e89a1","Type":"ContainerDied","Data":"5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f"} Jul 22 02:27:26.960564 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:26.960517 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5749f4067820fb1194f697f3c0ca0f30f924df14e277464c5cb44c43b8152e5f" Jul 22 02:27:27.088710 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.088680 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:27.117353 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.117325 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle\") pod \"0a33f8c1-112a-4035-898b-dab1d04d57fd\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " Jul 22 02:27:27.117501 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.117421 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vw5tt\" (UniqueName: \"kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt\") pod \"0a33f8c1-112a-4035-898b-dab1d04d57fd\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " Jul 22 02:27:27.117501 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.117442 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util\") pod \"0a33f8c1-112a-4035-898b-dab1d04d57fd\" (UID: \"0a33f8c1-112a-4035-898b-dab1d04d57fd\") " Jul 22 02:27:27.117984 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.117956 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle" (OuterVolumeSpecName: "bundle") pod "0a33f8c1-112a-4035-898b-dab1d04d57fd" (UID: "0a33f8c1-112a-4035-898b-dab1d04d57fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:27.119704 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.119675 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt" (OuterVolumeSpecName: "kube-api-access-vw5tt") pod "0a33f8c1-112a-4035-898b-dab1d04d57fd" (UID: "0a33f8c1-112a-4035-898b-dab1d04d57fd"). InnerVolumeSpecName "kube-api-access-vw5tt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:27:27.126330 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.126292 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util" (OuterVolumeSpecName: "util") pod "0a33f8c1-112a-4035-898b-dab1d04d57fd" (UID: "0a33f8c1-112a-4035-898b-dab1d04d57fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:27.218476 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.218440 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:27.218476 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.218469 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vw5tt\" (UniqueName: \"kubernetes.io/projected/0a33f8c1-112a-4035-898b-dab1d04d57fd-kube-api-access-vw5tt\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:27.218476 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.218481 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0a33f8c1-112a-4035-898b-dab1d04d57fd-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:27.967379 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.967351 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" Jul 22 02:27:27.967741 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.967384 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv" event={"ID":"0a33f8c1-112a-4035-898b-dab1d04d57fd","Type":"ContainerDied","Data":"b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf"} Jul 22 02:27:27.967741 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:27.967428 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b6cba814a128dec0d775422d9f970cec345ae1bf3c27e566dc54d7881f6566bf" Jul 22 02:27:28.099643 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.099621 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:28.124862 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.124840 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle\") pod \"5b89c749-870a-40fb-a56d-95f238ac322f\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " Jul 22 02:27:28.124993 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.124901 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x5rnq\" (UniqueName: \"kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq\") pod \"5b89c749-870a-40fb-a56d-95f238ac322f\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " Jul 22 02:27:28.124993 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.124969 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util\") pod \"5b89c749-870a-40fb-a56d-95f238ac322f\" (UID: \"5b89c749-870a-40fb-a56d-95f238ac322f\") " Jul 22 02:27:28.125472 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.125448 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle" (OuterVolumeSpecName: "bundle") pod "5b89c749-870a-40fb-a56d-95f238ac322f" (UID: "5b89c749-870a-40fb-a56d-95f238ac322f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:28.127035 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.127015 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq" (OuterVolumeSpecName: "kube-api-access-x5rnq") pod "5b89c749-870a-40fb-a56d-95f238ac322f" (UID: "5b89c749-870a-40fb-a56d-95f238ac322f"). InnerVolumeSpecName "kube-api-access-x5rnq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:27:28.133892 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.133866 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util" (OuterVolumeSpecName: "util") pod "5b89c749-870a-40fb-a56d-95f238ac322f" (UID: "5b89c749-870a-40fb-a56d-95f238ac322f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 22 02:27:28.226102 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.226023 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x5rnq\" (UniqueName: \"kubernetes.io/projected/5b89c749-870a-40fb-a56d-95f238ac322f-kube-api-access-x5rnq\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:28.226102 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.226050 2570 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-util\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:28.226102 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.226058 2570 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b89c749-870a-40fb-a56d-95f238ac322f-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:27:28.973546 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.973505 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" event={"ID":"5b89c749-870a-40fb-a56d-95f238ac322f","Type":"ContainerDied","Data":"af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b"} Jul 22 02:27:28.973916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.973556 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="af1ab85f13fc23309baec8abccb44b31db305c7f31db1b6bdfdee7a60c3d5d7b" Jul 22 02:27:28.973916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:28.973516 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr" Jul 22 02:27:30.321195 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321164 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f59dbdcbc-f97f6"] Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321566 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321581 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321596 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321604 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321619 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="pull" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321627 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="pull" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321642 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="util" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321650 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="util" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321660 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321667 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="extract" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321676 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="util" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321685 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="util" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321701 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="pull" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321708 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="pull" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321718 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="util" Jul 22 02:27:30.321717 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321726 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="util" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321735 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="util" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321743 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="util" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321754 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="pull" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321761 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="pull" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321769 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="extract" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321775 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="extract" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321786 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="pull" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321793 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="pull" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321865 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ed129b2-0873-4730-8076-e421f83e89a1" containerName="extract" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321877 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b89c749-870a-40fb-a56d-95f238ac322f" containerName="extract" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321885 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c99df528-8927-4321-989e-13bcfbe76134" containerName="extract" Jul 22 02:27:30.322280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.321894 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="0a33f8c1-112a-4035-898b-dab1d04d57fd" containerName="extract" Jul 22 02:27:30.325162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.325144 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.341302 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.341279 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f59dbdcbc-f97f6"] Jul 22 02:27:30.444909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.444865 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-oauth-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.444909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.444917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b4mh\" (UniqueName: \"kubernetes.io/projected/233cf869-8b3e-444f-a6b9-235e77ba927e-kube-api-access-6b4mh\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.445124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.444937 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-console-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.445124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.444966 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-oauth-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.445124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.445000 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-service-ca\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.445124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.445050 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.445124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.445065 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-trusted-ca-bundle\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546357 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546322 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546357 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546358 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-trusted-ca-bundle\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546383 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-oauth-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546405 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6b4mh\" (UniqueName: \"kubernetes.io/projected/233cf869-8b3e-444f-a6b9-235e77ba927e-kube-api-access-6b4mh\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546468 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-console-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546511 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-oauth-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.546617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.546559 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-service-ca\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.547260 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.547224 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-console-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.547260 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.547228 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-service-ca\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.547519 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.547495 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-oauth-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.547559 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.547544 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/233cf869-8b3e-444f-a6b9-235e77ba927e-trusted-ca-bundle\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.548920 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.548902 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-oauth-config\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.549145 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.549113 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/233cf869-8b3e-444f-a6b9-235e77ba927e-console-serving-cert\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.556043 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.556023 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b4mh\" (UniqueName: \"kubernetes.io/projected/233cf869-8b3e-444f-a6b9-235e77ba927e-kube-api-access-6b4mh\") pod \"console-5f59dbdcbc-f97f6\" (UID: \"233cf869-8b3e-444f-a6b9-235e77ba927e\") " pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.634185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.634082 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:30.766534 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.766510 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f59dbdcbc-f97f6"] Jul 22 02:27:30.767993 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:27:30.767963 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod233cf869_8b3e_444f_a6b9_235e77ba927e.slice/crio-1d922171ef51ebb816c20d68abafd2a532f886dcc0be59ce69d132cfb5679f7d WatchSource:0}: Error finding container 1d922171ef51ebb816c20d68abafd2a532f886dcc0be59ce69d132cfb5679f7d: Status 404 returned error can't find the container with id 1d922171ef51ebb816c20d68abafd2a532f886dcc0be59ce69d132cfb5679f7d Jul 22 02:27:30.984049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.984009 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59dbdcbc-f97f6" event={"ID":"233cf869-8b3e-444f-a6b9-235e77ba927e","Type":"ContainerStarted","Data":"6a80687110089c11e867bb21b596178d19cb7be67281f0f9dd2199b70fee2bde"} Jul 22 02:27:30.984204 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:30.984065 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f59dbdcbc-f97f6" event={"ID":"233cf869-8b3e-444f-a6b9-235e77ba927e","Type":"ContainerStarted","Data":"1d922171ef51ebb816c20d68abafd2a532f886dcc0be59ce69d132cfb5679f7d"} Jul 22 02:27:31.003489 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:31.003436 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f59dbdcbc-f97f6" podStartSLOduration=1.003420109 podStartE2EDuration="1.003420109s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:27:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:27:31.002153364 +0000 UTC m=+492.780229301" watchObservedRunningTime="2026-07-22 02:27:31.003420109 +0000 UTC m=+492.781496049" Jul 22 02:27:40.634574 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:40.634534 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:40.634939 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:40.634623 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:40.638964 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:40.638944 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:41.034109 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:41.034083 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f59dbdcbc-f97f6" Jul 22 02:27:41.147239 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:27:41.147209 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:28:06.167758 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.167697 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7dddb65df9-bz766" podUID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" containerName="console" containerID="cri-o://38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2" gracePeriod=15 Jul 22 02:28:06.402538 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.402513 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:28:06.559010 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.558975 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559019 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559046 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559072 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559373 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559235 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lvqnh\" (UniqueName: \"kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559373 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559294 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559373 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559328 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert\") pod \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\" (UID: \"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b\") " Jul 22 02:28:06.559503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559442 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:28:06.559503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559452 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config" (OuterVolumeSpecName: "console-config") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:28:06.559607 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559496 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:28:06.559776 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559751 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca" (OuterVolumeSpecName: "service-ca") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:28:06.559839 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559760 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-trusted-ca-bundle\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.559839 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559803 2570 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-oauth-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.559839 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.559820 2570 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.561489 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.561469 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:28:06.561580 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.561494 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh" (OuterVolumeSpecName: "kube-api-access-lvqnh") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "kube-api-access-lvqnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:06.561580 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.561509 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" (UID: "ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:28:06.660474 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.660437 2570 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.660474 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.660465 2570 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-serving-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.660474 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.660474 2570 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-console-oauth-config\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:06.660474 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:06.660483 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lvqnh\" (UniqueName: \"kubernetes.io/projected/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b-kube-api-access-lvqnh\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:07.155110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.155080 2570 generic.go:356] "Generic (PLEG): container finished" podID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" containerID="38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2" exitCode=0 Jul 22 02:28:07.155348 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.155120 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dddb65df9-bz766" event={"ID":"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b","Type":"ContainerDied","Data":"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2"} Jul 22 02:28:07.155348 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.155125 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7dddb65df9-bz766" Jul 22 02:28:07.155348 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.155182 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7dddb65df9-bz766" event={"ID":"ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b","Type":"ContainerDied","Data":"9276d78614607893af16d470e6f29ff51d99692cc77c0a8f0fa4695343bd434c"} Jul 22 02:28:07.155348 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.155202 2570 scope.go:117] "RemoveContainer" containerID="38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2" Jul 22 02:28:07.163389 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.163371 2570 scope.go:117] "RemoveContainer" containerID="38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2" Jul 22 02:28:07.163645 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:28:07.163624 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2\": container with ID starting with 38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2 not found: ID does not exist" containerID="38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2" Jul 22 02:28:07.163707 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.163652 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2"} err="failed to get container status \"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2\": rpc error: code = NotFound desc = could not find container \"38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2\": container with ID starting with 38be58fbd08f91ef2ae11748143b100188c43e5e0ab9fba2f6c4d7294e3ba7f2 not found: ID does not exist" Jul 22 02:28:07.186822 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.186792 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:28:07.193008 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:07.192989 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7dddb65df9-bz766"] Jul 22 02:28:08.748252 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:08.748219 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" path="/var/lib/kubelet/pods/ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b/volumes" Jul 22 02:28:36.107371 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.107338 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:36.108245 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.108218 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" containerName="console" Jul 22 02:28:36.108366 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.108249 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" containerName="console" Jul 22 02:28:36.108470 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.108455 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1e8bd1-2f4b-4c7b-9315-865ea1dcfe3b" containerName="console" Jul 22 02:28:36.116122 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.116095 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.120049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.120029 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-ss4lb\"" Jul 22 02:28:36.120498 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.120478 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 22 02:28:36.121107 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.121089 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 22 02:28:36.121227 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.121159 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 22 02:28:36.134670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.134641 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:36.188266 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.188237 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.188427 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.188309 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m49rd\" (UniqueName: \"kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.254226 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.254185 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:36.289681 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.289653 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.289828 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.289741 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m49rd\" (UniqueName: \"kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.290250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.290234 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.307527 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.307500 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m49rd\" (UniqueName: \"kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd\") pod \"limitador-limitador-64c94fd64f-bfwvg\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.427475 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.427397 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:36.604903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:36.604786 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:36.607067 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:28:36.607037 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbcadb7b_a3a8_4f98_ae63_5196725a0105.slice/crio-53c6edd618384541a3c8d86299876c85f68f205f485c770a73e6eb0d25957a83 WatchSource:0}: Error finding container 53c6edd618384541a3c8d86299876c85f68f205f485c770a73e6eb0d25957a83: Status 404 returned error can't find the container with id 53c6edd618384541a3c8d86299876c85f68f205f485c770a73e6eb0d25957a83 Jul 22 02:28:37.299741 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:37.299705 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" event={"ID":"dbcadb7b-a3a8-4f98-ae63-5196725a0105","Type":"ContainerStarted","Data":"53c6edd618384541a3c8d86299876c85f68f205f485c770a73e6eb0d25957a83"} Jul 22 02:28:39.312606 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:39.312580 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" event={"ID":"dbcadb7b-a3a8-4f98-ae63-5196725a0105","Type":"ContainerStarted","Data":"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5"} Jul 22 02:28:39.312965 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:39.312706 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:39.343263 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:39.343213 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" podStartSLOduration=0.719077641 podStartE2EDuration="3.343195723s" totalImagesPullingTime="2.624118082s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:28:39.339419587 +0000 UTC m=+561.117495526" watchObservedRunningTime="2026-07-22 02:28:39.343195723 +0000 UTC m=+561.121271662" Jul 22 02:28:50.318745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:50.318714 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:51.609882 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:51.609846 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:51.610358 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:51.610024 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" podUID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" containerName="limitador" containerID="cri-o://d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5" gracePeriod=30 Jul 22 02:28:52.143545 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.143524 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:52.219053 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.219020 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file\") pod \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " Jul 22 02:28:52.219240 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.219159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m49rd\" (UniqueName: \"kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd\") pod \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\" (UID: \"dbcadb7b-a3a8-4f98-ae63-5196725a0105\") " Jul 22 02:28:52.219451 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.219428 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file" (OuterVolumeSpecName: "config-file") pod "dbcadb7b-a3a8-4f98-ae63-5196725a0105" (UID: "dbcadb7b-a3a8-4f98-ae63-5196725a0105"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:28:52.221208 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.221189 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd" (OuterVolumeSpecName: "kube-api-access-m49rd") pod "dbcadb7b-a3a8-4f98-ae63-5196725a0105" (UID: "dbcadb7b-a3a8-4f98-ae63-5196725a0105"). InnerVolumeSpecName "kube-api-access-m49rd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:28:52.319702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.319669 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m49rd\" (UniqueName: \"kubernetes.io/projected/dbcadb7b-a3a8-4f98-ae63-5196725a0105-kube-api-access-m49rd\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:52.319702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.319697 2570 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/dbcadb7b-a3a8-4f98-ae63-5196725a0105-config-file\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:28:52.381584 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.381557 2570 generic.go:356] "Generic (PLEG): container finished" podID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" containerID="d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5" exitCode=0 Jul 22 02:28:52.381742 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.381595 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" event={"ID":"dbcadb7b-a3a8-4f98-ae63-5196725a0105","Type":"ContainerDied","Data":"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5"} Jul 22 02:28:52.381742 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.381608 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" Jul 22 02:28:52.381742 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.381631 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-bfwvg" event={"ID":"dbcadb7b-a3a8-4f98-ae63-5196725a0105","Type":"ContainerDied","Data":"53c6edd618384541a3c8d86299876c85f68f205f485c770a73e6eb0d25957a83"} Jul 22 02:28:52.381742 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.381656 2570 scope.go:117] "RemoveContainer" containerID="d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5" Jul 22 02:28:52.390645 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.390623 2570 scope.go:117] "RemoveContainer" containerID="d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5" Jul 22 02:28:52.390879 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:28:52.390863 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5\": container with ID starting with d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5 not found: ID does not exist" containerID="d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5" Jul 22 02:28:52.390937 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.390886 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5"} err="failed to get container status \"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5\": rpc error: code = NotFound desc = could not find container \"d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5\": container with ID starting with d5247fb34e299e7820b6fbbb731d8b3193bf6034a34b049b07a3795a9d15e6d5 not found: ID does not exist" Jul 22 02:28:52.405101 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.405076 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:52.408518 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.408495 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-bfwvg"] Jul 22 02:28:52.753616 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:52.750665 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" path="/var/lib/kubelet/pods/dbcadb7b-a3a8-4f98-ae63-5196725a0105/volumes" Jul 22 02:28:57.696111 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.696079 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg"] Jul 22 02:28:57.696656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.696531 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" containerName="limitador" Jul 22 02:28:57.696656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.696549 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" containerName="limitador" Jul 22 02:28:57.696656 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.696638 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="dbcadb7b-a3a8-4f98-ae63-5196725a0105" containerName="limitador" Jul 22 02:28:57.701026 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.701002 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.703729 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.703709 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 22 02:28:57.703836 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.703813 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-x878k\"" Jul 22 02:28:57.704792 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.704777 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 22 02:28:57.704876 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.704798 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 22 02:28:57.708797 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.708507 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg"] Jul 22 02:28:57.766075 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.766044 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9815bb5-988b-400e-b9e9-30bd59757e58-data\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.766287 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.766164 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t42gm\" (UniqueName: \"kubernetes.io/projected/e9815bb5-988b-400e-b9e9-30bd59757e58-kube-api-access-t42gm\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.866587 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.866553 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t42gm\" (UniqueName: \"kubernetes.io/projected/e9815bb5-988b-400e-b9e9-30bd59757e58-kube-api-access-t42gm\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.866759 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.866598 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9815bb5-988b-400e-b9e9-30bd59757e58-data\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.866924 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.866910 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/e9815bb5-988b-400e-b9e9-30bd59757e58-data\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:57.881429 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:57.881398 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t42gm\" (UniqueName: \"kubernetes.io/projected/e9815bb5-988b-400e-b9e9-30bd59757e58-kube-api-access-t42gm\") pod \"postgres-69fb4594cf-7m2qg\" (UID: \"e9815bb5-988b-400e-b9e9-30bd59757e58\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:58.012150 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:58.012107 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:28:58.150279 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:58.150256 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg"] Jul 22 02:28:58.152879 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:28:58.152853 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9815bb5_988b_400e_b9e9_30bd59757e58.slice/crio-c0d8c72679168dc3137f9f5d67181bf20ce928ae1f361758f78c268bd4ba7d61 WatchSource:0}: Error finding container c0d8c72679168dc3137f9f5d67181bf20ce928ae1f361758f78c268bd4ba7d61: Status 404 returned error can't find the container with id c0d8c72679168dc3137f9f5d67181bf20ce928ae1f361758f78c268bd4ba7d61 Jul 22 02:28:58.412061 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:28:58.411970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" event={"ID":"e9815bb5-988b-400e-b9e9-30bd59757e58","Type":"ContainerStarted","Data":"c0d8c72679168dc3137f9f5d67181bf20ce928ae1f361758f78c268bd4ba7d61"} Jul 22 02:29:18.678944 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:18.678918 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:29:18.683571 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:18.683537 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:29:18.691716 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:18.691697 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:29:18.696623 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:18.696607 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:29:26.037966 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:26.037941 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 22 02:29:26.552049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:26.552015 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" event={"ID":"e9815bb5-988b-400e-b9e9-30bd59757e58","Type":"ContainerStarted","Data":"8b4e84a7721dd82c58db24f75792472db829c84e78f8b5598f6f3e9851657bd6"} Jul 22 02:29:26.552280 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:26.552178 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:29:26.571691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:26.571639 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" podStartSLOduration=1.690502231 podStartE2EDuration="29.571618504s" totalImagesPullingTime="27.881116273s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:28:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:26.569849461 +0000 UTC m=+608.347925401" watchObservedRunningTime="2026-07-22 02:29:26.571618504 +0000 UTC m=+608.349694445" Jul 22 02:29:32.586352 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:32.586326 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-7m2qg" Jul 22 02:29:39.839241 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.839153 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:39.844874 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.844854 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:39.847850 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.847830 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wcpnl\"" Jul 22 02:29:39.847956 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.847895 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 02:29:39.848820 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.848804 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 02:29:39.855447 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.855422 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:39.931342 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.931312 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:39.931496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:39.931368 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sc8vz\" (UniqueName: \"kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.030767 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.030735 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:29:40.031996 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.031967 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sc8vz\" (UniqueName: \"kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.032123 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.032084 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.034640 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.034614 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.038364 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.038343 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.047490 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.047266 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:29:40.050638 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.050618 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sc8vz\" (UniqueName: \"kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz\") pod \"maas-controller-867559458b-7zpkp\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.133445 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.133369 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntkzp\" (UniqueName: \"kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.133445 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.133406 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.155325 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.155298 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:40.234811 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.234777 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.234974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.234853 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntkzp\" (UniqueName: \"kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.237722 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.237700 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.244555 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.244529 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntkzp\" (UniqueName: \"kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp\") pod \"maas-controller-894c47d6f-7xwp6\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.288924 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.288896 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:40.291766 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:29:40.291744 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10152493_0334_4b7e_bffd_4960bda3e7d2.slice/crio-415e3f61e29ffaf505e6f1023b56ed40a4d223632190e6fb6a9376e05a3345ad WatchSource:0}: Error finding container 415e3f61e29ffaf505e6f1023b56ed40a4d223632190e6fb6a9376e05a3345ad: Status 404 returned error can't find the container with id 415e3f61e29ffaf505e6f1023b56ed40a4d223632190e6fb6a9376e05a3345ad Jul 22 02:29:40.352465 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.352435 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:40.487375 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.487340 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:29:40.489957 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:29:40.489928 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5357c4a7_d71b_42d4_9ef5_21268666d90d.slice/crio-b63a0500df97947b923131e29081b1349d799ee5cba5bf9de3d928744e1aaacd WatchSource:0}: Error finding container b63a0500df97947b923131e29081b1349d799ee5cba5bf9de3d928744e1aaacd: Status 404 returned error can't find the container with id b63a0500df97947b923131e29081b1349d799ee5cba5bf9de3d928744e1aaacd Jul 22 02:29:40.618667 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.618635 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-7xwp6" event={"ID":"5357c4a7-d71b-42d4-9ef5-21268666d90d","Type":"ContainerStarted","Data":"b63a0500df97947b923131e29081b1349d799ee5cba5bf9de3d928744e1aaacd"} Jul 22 02:29:40.619736 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:40.619714 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-867559458b-7zpkp" event={"ID":"10152493-0334-4b7e-bffd-4960bda3e7d2","Type":"ContainerStarted","Data":"415e3f61e29ffaf505e6f1023b56ed40a4d223632190e6fb6a9376e05a3345ad"} Jul 22 02:29:43.639371 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.639337 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-867559458b-7zpkp" event={"ID":"10152493-0334-4b7e-bffd-4960bda3e7d2","Type":"ContainerStarted","Data":"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b"} Jul 22 02:29:43.639787 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.639403 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:43.640756 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.640738 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-7xwp6" event={"ID":"5357c4a7-d71b-42d4-9ef5-21268666d90d","Type":"ContainerStarted","Data":"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb"} Jul 22 02:29:43.640854 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.640777 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:43.679161 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.679101 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-867559458b-7zpkp" podStartSLOduration=2.1918622389999998 podStartE2EDuration="4.679085274s" totalImagesPullingTime="2.487223035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:43.677752547 +0000 UTC m=+625.455828487" watchObservedRunningTime="2026-07-22 02:29:43.679085274 +0000 UTC m=+625.457161213" Jul 22 02:29:43.730235 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:43.730121 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-894c47d6f-7xwp6" podStartSLOduration=1.430396294 podStartE2EDuration="3.730104325s" totalImagesPullingTime="2.299708031s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:29:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:29:43.72804695 +0000 UTC m=+625.506122892" watchObservedRunningTime="2026-07-22 02:29:43.730104325 +0000 UTC m=+625.508180295" Jul 22 02:29:54.648029 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:54.647998 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:29:54.648441 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:54.648084 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:54.720873 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:54.720840 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:54.721126 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:54.721066 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-867559458b-7zpkp" podUID="10152493-0334-4b7e-bffd-4960bda3e7d2" containerName="manager" containerID="cri-o://a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b" gracePeriod=30 Jul 22 02:29:54.960785 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:54.960763 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:55.068170 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.068114 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sc8vz\" (UniqueName: \"kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz\") pod \"10152493-0334-4b7e-bffd-4960bda3e7d2\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " Jul 22 02:29:55.068170 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.068176 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert\") pod \"10152493-0334-4b7e-bffd-4960bda3e7d2\" (UID: \"10152493-0334-4b7e-bffd-4960bda3e7d2\") " Jul 22 02:29:55.070499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.070472 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz" (OuterVolumeSpecName: "kube-api-access-sc8vz") pod "10152493-0334-4b7e-bffd-4960bda3e7d2" (UID: "10152493-0334-4b7e-bffd-4960bda3e7d2"). InnerVolumeSpecName "kube-api-access-sc8vz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:29:55.070609 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.070494 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "10152493-0334-4b7e-bffd-4960bda3e7d2" (UID: "10152493-0334-4b7e-bffd-4960bda3e7d2"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:29:55.169328 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.169284 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sc8vz\" (UniqueName: \"kubernetes.io/projected/10152493-0334-4b7e-bffd-4960bda3e7d2-kube-api-access-sc8vz\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:29:55.169328 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.169323 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10152493-0334-4b7e-bffd-4960bda3e7d2-webhook-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:29:55.701901 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.701866 2570 generic.go:356] "Generic (PLEG): container finished" podID="10152493-0334-4b7e-bffd-4960bda3e7d2" containerID="a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b" exitCode=0 Jul 22 02:29:55.702323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.701909 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-867559458b-7zpkp" Jul 22 02:29:55.702323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.701958 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-867559458b-7zpkp" event={"ID":"10152493-0334-4b7e-bffd-4960bda3e7d2","Type":"ContainerDied","Data":"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b"} Jul 22 02:29:55.702323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.702006 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-867559458b-7zpkp" event={"ID":"10152493-0334-4b7e-bffd-4960bda3e7d2","Type":"ContainerDied","Data":"415e3f61e29ffaf505e6f1023b56ed40a4d223632190e6fb6a9376e05a3345ad"} Jul 22 02:29:55.702323 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.702027 2570 scope.go:117] "RemoveContainer" containerID="a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b" Jul 22 02:29:55.711076 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.711056 2570 scope.go:117] "RemoveContainer" containerID="a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b" Jul 22 02:29:55.711351 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:29:55.711335 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b\": container with ID starting with a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b not found: ID does not exist" containerID="a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b" Jul 22 02:29:55.711403 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.711361 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b"} err="failed to get container status \"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b\": rpc error: code = NotFound desc = could not find container \"a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b\": container with ID starting with a32c4d979f714e5db871512361c7df87c649c3e4417d273ded98beda7a31491b not found: ID does not exist" Jul 22 02:29:55.724466 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.724442 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:55.728452 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:55.728431 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-867559458b-7zpkp"] Jul 22 02:29:56.748212 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:29:56.748180 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="10152493-0334-4b7e-bffd-4960bda3e7d2" path="/var/lib/kubelet/pods/10152493-0334-4b7e-bffd-4960bda3e7d2/volumes" Jul 22 02:30:16.599175 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.599125 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-79f97668bb-v782h"] Jul 22 02:30:16.599830 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.599464 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10152493-0334-4b7e-bffd-4960bda3e7d2" containerName="manager" Jul 22 02:30:16.599830 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.599479 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="10152493-0334-4b7e-bffd-4960bda3e7d2" containerName="manager" Jul 22 02:30:16.599830 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.599547 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="10152493-0334-4b7e-bffd-4960bda3e7d2" containerName="manager" Jul 22 02:30:16.602424 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.602408 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.607226 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.607207 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 22 02:30:16.617781 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.617757 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79f97668bb-v782h"] Jul 22 02:30:16.641749 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.641719 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bmhk6\" (UniqueName: \"kubernetes.io/projected/fd4e218d-c483-4b50-84b0-7649b47f7314-kube-api-access-bmhk6\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.641896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.641763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4e218d-c483-4b50-84b0-7649b47f7314-openshift-service-ca\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.641896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.641789 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.742495 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.742463 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bmhk6\" (UniqueName: \"kubernetes.io/projected/fd4e218d-c483-4b50-84b0-7649b47f7314-kube-api-access-bmhk6\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.742673 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.742518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4e218d-c483-4b50-84b0-7649b47f7314-openshift-service-ca\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.742673 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.742551 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.742673 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:30:16.742666 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 22 02:30:16.742841 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:30:16.742719 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls podName:fd4e218d-c483-4b50-84b0-7649b47f7314 nodeName:}" failed. No retries permitted until 2026-07-22 02:30:17.242703016 +0000 UTC m=+659.020778934 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls") pod "maas-api-79f97668bb-v782h" (UID: "fd4e218d-c483-4b50-84b0-7649b47f7314") : secret "maas-api-serving-cert" not found Jul 22 02:30:16.743250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.743232 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fd4e218d-c483-4b50-84b0-7649b47f7314-openshift-service-ca\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:16.752527 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:16.752481 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bmhk6\" (UniqueName: \"kubernetes.io/projected/fd4e218d-c483-4b50-84b0-7649b47f7314-kube-api-access-bmhk6\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:17.247206 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.247157 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:17.249600 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.249578 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fd4e218d-c483-4b50-84b0-7649b47f7314-maas-api-tls\") pod \"maas-api-79f97668bb-v782h\" (UID: \"fd4e218d-c483-4b50-84b0-7649b47f7314\") " pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:17.262233 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.262212 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dd2b2"] Jul 22 02:30:17.270434 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.270418 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.273750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.273728 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 22 02:30:17.292967 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.292941 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4zcd7"] Jul 22 02:30:17.296908 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.296889 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dd2b2"] Jul 22 02:30:17.297017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.296973 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.302054 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:30:17.302021 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-dockercfg-2gzgm\" is forbidden: User \"system:node:ip-10-0-142-149.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-149.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2gzgm\"" type="*v1.Secret" Jul 22 02:30:17.347595 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.347566 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d73cb06-f4df-4793-a376-a3f93ee60f9f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.347750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.347613 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqx4f\" (UniqueName: \"kubernetes.io/projected/2d73cb06-f4df-4793-a376-a3f93ee60f9f-kube-api-access-sqx4f\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.347750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.347657 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cz2xc\" (UniqueName: \"kubernetes.io/projected/40a170f6-1f95-47bb-9b89-ff523bd57838-kube-api-access-cz2xc\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.347750 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.347715 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a170f6-1f95-47bb-9b89-ff523bd57838-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.355825 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.355798 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4zcd7"] Jul 22 02:30:17.448401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.448363 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a170f6-1f95-47bb-9b89-ff523bd57838-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.448597 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.448423 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d73cb06-f4df-4793-a376-a3f93ee60f9f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.448597 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.448483 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqx4f\" (UniqueName: \"kubernetes.io/projected/2d73cb06-f4df-4793-a376-a3f93ee60f9f-kube-api-access-sqx4f\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.448597 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.448518 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cz2xc\" (UniqueName: \"kubernetes.io/projected/40a170f6-1f95-47bb-9b89-ff523bd57838-kube-api-access-cz2xc\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.449171 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.449119 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/40a170f6-1f95-47bb-9b89-ff523bd57838-plugins-config-volume\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.449221 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.449173 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d73cb06-f4df-4793-a376-a3f93ee60f9f-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.475614 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.475585 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cz2xc\" (UniqueName: \"kubernetes.io/projected/40a170f6-1f95-47bb-9b89-ff523bd57838-kube-api-access-cz2xc\") pod \"payload-processing-5bf47cd988-dd2b2\" (UID: \"40a170f6-1f95-47bb-9b89-ff523bd57838\") " pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.478209 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.478186 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqx4f\" (UniqueName: \"kubernetes.io/projected/2d73cb06-f4df-4793-a376-a3f93ee60f9f-kube-api-access-sqx4f\") pod \"payload-pre-processing-7677b774bb-4zcd7\" (UID: \"2d73cb06-f4df-4793-a376-a3f93ee60f9f\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:17.511945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.511874 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:17.580205 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.580175 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:17.668947 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.668915 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-79f97668bb-v782h"] Jul 22 02:30:17.671503 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:30:17.671478 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd4e218d_c483_4b50_84b0_7649b47f7314.slice/crio-14e89820e41405848706487c6aa4b9fa988ef3b4c9ec4920513cbd7e39be62d1 WatchSource:0}: Error finding container 14e89820e41405848706487c6aa4b9fa988ef3b4c9ec4920513cbd7e39be62d1: Status 404 returned error can't find the container with id 14e89820e41405848706487c6aa4b9fa988ef3b4c9ec4920513cbd7e39be62d1 Jul 22 02:30:17.672670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.672653 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:30:17.730708 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.730683 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-dd2b2"] Jul 22 02:30:17.732039 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:30:17.732013 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40a170f6_1f95_47bb_9b89_ff523bd57838.slice/crio-dc25f4a45f5716bfdad011fb1035caf602373592cb2ee062983b8f76b95f60d8 WatchSource:0}: Error finding container dc25f4a45f5716bfdad011fb1035caf602373592cb2ee062983b8f76b95f60d8: Status 404 returned error can't find the container with id dc25f4a45f5716bfdad011fb1035caf602373592cb2ee062983b8f76b95f60d8 Jul 22 02:30:17.813609 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.813528 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" event={"ID":"fd4e218d-c483-4b50-84b0-7649b47f7314","Type":"ContainerStarted","Data":"14e89820e41405848706487c6aa4b9fa988ef3b4c9ec4920513cbd7e39be62d1"} Jul 22 02:30:17.814784 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:17.814740 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" event={"ID":"40a170f6-1f95-47bb-9b89-ff523bd57838","Type":"ContainerStarted","Data":"dc25f4a45f5716bfdad011fb1035caf602373592cb2ee062983b8f76b95f60d8"} Jul 22 02:30:18.553125 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:18.553087 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-2gzgm\"" Jul 22 02:30:18.559164 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:18.559115 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:18.748534 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:30:18.748504 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d73cb06_f4df_4793_a376_a3f93ee60f9f.slice/crio-81b5ec24ed48a021f6ce662d519e22e04b0316eb5fdd984454344b0c78f7da49 WatchSource:0}: Error finding container 81b5ec24ed48a021f6ce662d519e22e04b0316eb5fdd984454344b0c78f7da49: Status 404 returned error can't find the container with id 81b5ec24ed48a021f6ce662d519e22e04b0316eb5fdd984454344b0c78f7da49 Jul 22 02:30:18.754775 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:18.754749 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-4zcd7"] Jul 22 02:30:18.825465 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:18.825384 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" event={"ID":"2d73cb06-f4df-4793-a376-a3f93ee60f9f","Type":"ContainerStarted","Data":"81b5ec24ed48a021f6ce662d519e22e04b0316eb5fdd984454344b0c78f7da49"} Jul 22 02:30:19.832225 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.832193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" event={"ID":"40a170f6-1f95-47bb-9b89-ff523bd57838","Type":"ContainerStarted","Data":"f328ac8576a555e6cada78ee992f585939712a10a9e9ee5a0f06e5da8a1195ff"} Jul 22 02:30:19.832682 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.832312 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:19.833843 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.833817 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" event={"ID":"fd4e218d-c483-4b50-84b0-7649b47f7314","Type":"ContainerStarted","Data":"4f46aba431097a3806d4e0f3adb6e98b991e45a7b8384ed22d5ef2a390d0a81a"} Jul 22 02:30:19.833963 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.833919 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:19.835252 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.835233 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" event={"ID":"2d73cb06-f4df-4793-a376-a3f93ee60f9f","Type":"ContainerStarted","Data":"21ca27d319f09efeab365c339a2b827270ebfb80dc74d9d3602ab9f5ff9497a5"} Jul 22 02:30:19.835352 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.835284 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:30:19.848086 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.848042 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" podStartSLOduration=0.956681737 podStartE2EDuration="2.848027376s" totalImagesPullingTime="1.891345639s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:30:19.847979202 +0000 UTC m=+661.626055143" watchObservedRunningTime="2026-07-22 02:30:19.848027376 +0000 UTC m=+661.626103316" Jul 22 02:30:19.866612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.866521 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" podStartSLOduration=1.9958885450000001 podStartE2EDuration="2.866507112s" totalImagesPullingTime="870.618567ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:30:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:30:19.864167203 +0000 UTC m=+661.642243140" watchObservedRunningTime="2026-07-22 02:30:19.866507112 +0000 UTC m=+661.644583052" Jul 22 02:30:19.882528 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:19.882489 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" podStartSLOduration=1.929997121 podStartE2EDuration="3.882475717s" totalImagesPullingTime="1.952478596s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:30:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:30:19.879977826 +0000 UTC m=+661.658053765" watchObservedRunningTime="2026-07-22 02:30:19.882475717 +0000 UTC m=+661.660551656" Jul 22 02:30:25.846219 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:25.846192 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-79f97668bb-v782h" Jul 22 02:30:30.842347 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:30.842314 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-dd2b2" Jul 22 02:30:30.842875 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:30:30.842518 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-4zcd7" Jul 22 02:31:02.863983 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.863952 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb"] Jul 22 02:31:02.868857 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.868836 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.871807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.871761 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4mjch\"" Jul 22 02:31:02.871953 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.871838 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 22 02:31:02.872781 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.872761 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 22 02:31:02.872918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.872899 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 22 02:31:02.883103 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.883083 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb"] Jul 22 02:31:02.943910 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.943878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84cvx\" (UniqueName: \"kubernetes.io/projected/bacded41-e287-44f9-bc1d-f9f79ec90525-kube-api-access-84cvx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.943910 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.943914 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bacded41-e287-44f9-bc1d-f9f79ec90525-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.944163 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.943947 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.944163 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.943989 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.944163 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.944017 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:02.944163 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:02.944093 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045441 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045407 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045485 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84cvx\" (UniqueName: \"kubernetes.io/projected/bacded41-e287-44f9-bc1d-f9f79ec90525-kube-api-access-84cvx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bacded41-e287-44f9-bc1d-f9f79ec90525-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045550 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045572 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045627 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045919 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045855 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045919 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045907 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.045994 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.045934 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.047872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.047849 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bacded41-e287-44f9-bc1d-f9f79ec90525-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.048262 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.048240 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bacded41-e287-44f9-bc1d-f9f79ec90525-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.055407 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.055388 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84cvx\" (UniqueName: \"kubernetes.io/projected/bacded41-e287-44f9-bc1d-f9f79ec90525-kube-api-access-84cvx\") pod \"e2e-distinct-simulated-kserve-59975d69f9-qt2qb\" (UID: \"bacded41-e287-44f9-bc1d-f9f79ec90525\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.179516 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.179426 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:03.317320 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:03.317292 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb"] Jul 22 02:31:03.318105 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:31:03.318084 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbacded41_e287_44f9_bc1d_f9f79ec90525.slice/crio-a7a8df7916852df84cdba67ec7a8a2257031c6b61c4da531a96e048339c06871 WatchSource:0}: Error finding container a7a8df7916852df84cdba67ec7a8a2257031c6b61c4da531a96e048339c06871: Status 404 returned error can't find the container with id a7a8df7916852df84cdba67ec7a8a2257031c6b61c4da531a96e048339c06871 Jul 22 02:31:04.063761 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:04.063723 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" event={"ID":"bacded41-e287-44f9-bc1d-f9f79ec90525","Type":"ContainerStarted","Data":"a7a8df7916852df84cdba67ec7a8a2257031c6b61c4da531a96e048339c06871"} Jul 22 02:31:05.070037 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:05.069997 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" event={"ID":"bacded41-e287-44f9-bc1d-f9f79ec90525","Type":"ContainerStarted","Data":"3c5c740b06007c47a700a008fa66c735eba95178f329b14bcb777ee7c9c882c5"} Jul 22 02:31:05.098792 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:05.098719 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" podStartSLOduration=2.06879746 podStartE2EDuration="3.09870145s" totalImagesPullingTime="1.02990399s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:05.098214369 +0000 UTC m=+706.876290313" watchObservedRunningTime="2026-07-22 02:31:05.09870145 +0000 UTC m=+706.876777389" Jul 22 02:31:13.179929 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:13.179843 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:13.179929 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:13.179910 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:13.192194 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:13.192169 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:14.127531 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:14.127498 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-qt2qb" Jul 22 02:31:19.052268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.052235 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz"] Jul 22 02:31:19.055620 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.055602 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.058343 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.058322 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 22 02:31:19.068964 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.068940 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz"] Jul 22 02:31:19.188363 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188334 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.188550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188378 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mz6bq\" (UniqueName: \"kubernetes.io/projected/386df561-d012-4c66-8891-a17970772fb2-kube-api-access-mz6bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.188550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188397 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.188550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188474 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.188550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188514 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386df561-d012-4c66-8891-a17970772fb2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.188550 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.188539 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289761 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mz6bq\" (UniqueName: \"kubernetes.io/projected/386df561-d012-4c66-8891-a17970772fb2-kube-api-access-mz6bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289778 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289807 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289832 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386df561-d012-4c66-8891-a17970772fb2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.289918 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.289852 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.290268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.290198 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.290268 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.290255 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.290380 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.290342 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.292148 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.292113 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/386df561-d012-4c66-8891-a17970772fb2-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.292381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.292364 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/386df561-d012-4c66-8891-a17970772fb2-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.300658 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.300635 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mz6bq\" (UniqueName: \"kubernetes.io/projected/386df561-d012-4c66-8891-a17970772fb2-kube-api-access-mz6bq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz\" (UID: \"386df561-d012-4c66-8891-a17970772fb2\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.365754 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.365663 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:19.502177 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:19.500448 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz"] Jul 22 02:31:19.503100 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:31:19.503063 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod386df561_d012_4c66_8891_a17970772fb2.slice/crio-081fb440f4f9dbf1fc34a5325c1da25ce17239f4a72135db2a8e52836dfa0676 WatchSource:0}: Error finding container 081fb440f4f9dbf1fc34a5325c1da25ce17239f4a72135db2a8e52836dfa0676: Status 404 returned error can't find the container with id 081fb440f4f9dbf1fc34a5325c1da25ce17239f4a72135db2a8e52836dfa0676 Jul 22 02:31:20.146724 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:20.146627 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" event={"ID":"386df561-d012-4c66-8891-a17970772fb2","Type":"ContainerStarted","Data":"6e1b6a9eb301c15ac1f671b2ee8d6c81f97ddf32492b9b93f6e2b9e388ea07fd"} Jul 22 02:31:20.146724 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:20.146692 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" event={"ID":"386df561-d012-4c66-8891-a17970772fb2","Type":"ContainerStarted","Data":"081fb440f4f9dbf1fc34a5325c1da25ce17239f4a72135db2a8e52836dfa0676"} Jul 22 02:31:20.168153 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:20.168081 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" podStartSLOduration=0.853697128 podStartE2EDuration="1.168065603s" totalImagesPullingTime="314.368475ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:31:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:31:20.16642092 +0000 UTC m=+721.944496862" watchObservedRunningTime="2026-07-22 02:31:20.168065603 +0000 UTC m=+721.946141595" Jul 22 02:31:29.366317 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:29.366277 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:29.366711 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:29.366350 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:29.378530 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:29.378500 2570 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:31:30.213874 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:31:30.213851 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz" Jul 22 02:33:22.423809 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.423759 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:33:22.424340 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.424011 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-894c47d6f-7xwp6" podUID="5357c4a7-d71b-42d4-9ef5-21268666d90d" containerName="manager" containerID="cri-o://2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb" gracePeriod=30 Jul 22 02:33:22.681084 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.681013 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:33:22.784194 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.784128 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert\") pod \"5357c4a7-d71b-42d4-9ef5-21268666d90d\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " Jul 22 02:33:22.784399 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.784217 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ntkzp\" (UniqueName: \"kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp\") pod \"5357c4a7-d71b-42d4-9ef5-21268666d90d\" (UID: \"5357c4a7-d71b-42d4-9ef5-21268666d90d\") " Jul 22 02:33:22.786830 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.786794 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "5357c4a7-d71b-42d4-9ef5-21268666d90d" (UID: "5357c4a7-d71b-42d4-9ef5-21268666d90d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:33:22.786979 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.786959 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp" (OuterVolumeSpecName: "kube-api-access-ntkzp") pod "5357c4a7-d71b-42d4-9ef5-21268666d90d" (UID: "5357c4a7-d71b-42d4-9ef5-21268666d90d"). InnerVolumeSpecName "kube-api-access-ntkzp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:33:22.790903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.790872 2570 generic.go:356] "Generic (PLEG): container finished" podID="5357c4a7-d71b-42d4-9ef5-21268666d90d" containerID="2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb" exitCode=0 Jul 22 02:33:22.791049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.790936 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-7xwp6" Jul 22 02:33:22.791049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.790950 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-7xwp6" event={"ID":"5357c4a7-d71b-42d4-9ef5-21268666d90d","Type":"ContainerDied","Data":"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb"} Jul 22 02:33:22.791049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.791010 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-7xwp6" event={"ID":"5357c4a7-d71b-42d4-9ef5-21268666d90d","Type":"ContainerDied","Data":"b63a0500df97947b923131e29081b1349d799ee5cba5bf9de3d928744e1aaacd"} Jul 22 02:33:22.791049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.791034 2570 scope.go:117] "RemoveContainer" containerID="2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb" Jul 22 02:33:22.804849 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.804825 2570 scope.go:117] "RemoveContainer" containerID="2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb" Jul 22 02:33:22.805213 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:33:22.805183 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb\": container with ID starting with 2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb not found: ID does not exist" containerID="2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb" Jul 22 02:33:22.805289 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.805227 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb"} err="failed to get container status \"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb\": rpc error: code = NotFound desc = could not find container \"2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb\": container with ID starting with 2a24115ed4c30ca43247d8c233345bd1a865a23a44cefa4778bd7e5c4d6447eb not found: ID does not exist" Jul 22 02:33:22.817056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.817026 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:33:22.821743 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.821711 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-894c47d6f-7xwp6"] Jul 22 02:33:22.885635 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.885605 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/5357c4a7-d71b-42d4-9ef5-21268666d90d-webhook-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:33:22.885635 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:22.885634 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ntkzp\" (UniqueName: \"kubernetes.io/projected/5357c4a7-d71b-42d4-9ef5-21268666d90d-kube-api-access-ntkzp\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:33:24.749284 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:33:24.749245 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5357c4a7-d71b-42d4-9ef5-21268666d90d" path="/var/lib/kubelet/pods/5357c4a7-d71b-42d4-9ef5-21268666d90d/volumes" Jul 22 02:34:18.719819 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:18.719737 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:34:18.722362 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:18.722329 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:34:18.735561 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:18.735531 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:34:18.738287 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:18.738267 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:34:23.684422 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.684376 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:23.684930 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.684759 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5357c4a7-d71b-42d4-9ef5-21268666d90d" containerName="manager" Jul 22 02:34:23.684930 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.684773 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5357c4a7-d71b-42d4-9ef5-21268666d90d" containerName="manager" Jul 22 02:34:23.684930 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.684850 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5357c4a7-d71b-42d4-9ef5-21268666d90d" containerName="manager" Jul 22 02:34:23.689676 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.689650 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.693685 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.693658 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wcpnl\"" Jul 22 02:34:23.693853 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.693698 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 02:34:23.693853 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.693701 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 02:34:23.703556 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.703529 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:23.803903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.803862 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-crzbz\" (UniqueName: \"kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.804078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.803940 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.905334 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.905301 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-crzbz\" (UniqueName: \"kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.905499 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.905362 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.908046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.908013 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:23.914814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:23.914792 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-crzbz\" (UniqueName: \"kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz\") pod \"maas-controller-894c47d6f-jndrl\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:24.000318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:24.000283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:24.145169 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:24.143603 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:25.116361 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:25.116324 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-jndrl" event={"ID":"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce","Type":"ContainerStarted","Data":"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9"} Jul 22 02:34:25.116809 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:25.116387 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:25.116809 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:25.116405 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-jndrl" event={"ID":"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce","Type":"ContainerStarted","Data":"2bf7b179e7a29cdd0c79e8aac8d8a3d60f168c09f228a26a1d389ea99cc94136"} Jul 22 02:34:25.137895 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:25.137839 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-894c47d6f-jndrl" podStartSLOduration=1.753290159 podStartE2EDuration="2.137823394s" totalImagesPullingTime="384.533235ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:34:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:34:25.1363299 +0000 UTC m=+906.914405843" watchObservedRunningTime="2026-07-22 02:34:25.137823394 +0000 UTC m=+906.915899335" Jul 22 02:34:36.125730 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:36.125695 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:40.818473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:40.818434 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:40.818903 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:40.818643 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-894c47d6f-jndrl" podUID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" containerName="manager" containerID="cri-o://073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9" gracePeriod=30 Jul 22 02:34:41.060705 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.060682 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:41.149435 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.149281 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert\") pod \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " Jul 22 02:34:41.149610 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.149440 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-crzbz\" (UniqueName: \"kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz\") pod \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\" (UID: \"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce\") " Jul 22 02:34:41.151508 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.151476 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz" (OuterVolumeSpecName: "kube-api-access-crzbz") pod "ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" (UID: "ee6349a8-a2c4-40a4-b813-0b8df5baa3ce"). InnerVolumeSpecName "kube-api-access-crzbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:34:41.151593 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.151532 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" (UID: "ee6349a8-a2c4-40a4-b813-0b8df5baa3ce"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:34:41.199206 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.199173 2570 generic.go:356] "Generic (PLEG): container finished" podID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" containerID="073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9" exitCode=0 Jul 22 02:34:41.199378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.199219 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-jndrl" Jul 22 02:34:41.199378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.199262 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-jndrl" event={"ID":"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce","Type":"ContainerDied","Data":"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9"} Jul 22 02:34:41.199378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.199316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-jndrl" event={"ID":"ee6349a8-a2c4-40a4-b813-0b8df5baa3ce","Type":"ContainerDied","Data":"2bf7b179e7a29cdd0c79e8aac8d8a3d60f168c09f228a26a1d389ea99cc94136"} Jul 22 02:34:41.199378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.199341 2570 scope.go:117] "RemoveContainer" containerID="073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9" Jul 22 02:34:41.210151 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.210114 2570 scope.go:117] "RemoveContainer" containerID="073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9" Jul 22 02:34:41.210451 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:34:41.210433 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9\": container with ID starting with 073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9 not found: ID does not exist" containerID="073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9" Jul 22 02:34:41.210528 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.210462 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9"} err="failed to get container status \"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9\": rpc error: code = NotFound desc = could not find container \"073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9\": container with ID starting with 073e1ec11722aeb314af7ea1d027779ef0c6b33af566ac8deb5aa1b95f3824d9 not found: ID does not exist" Jul 22 02:34:41.226082 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.226046 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:41.230803 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.230778 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-894c47d6f-jndrl"] Jul 22 02:34:41.236437 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:34:41.236409 2570 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee6349a8_a2c4_40a4_b813_0b8df5baa3ce.slice\": RecentStats: unable to find data in memory cache]" Jul 22 02:34:41.250589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.250563 2570 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-webhook-cert\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:34:41.250589 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:41.250588 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-crzbz\" (UniqueName: \"kubernetes.io/projected/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce-kube-api-access-crzbz\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:34:42.070047 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.070013 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-894c47d6f-rsp4q"] Jul 22 02:34:42.070449 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.070419 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" containerName="manager" Jul 22 02:34:42.070449 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.070435 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" containerName="manager" Jul 22 02:34:42.070527 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.070497 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" containerName="manager" Jul 22 02:34:42.073385 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.073369 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.076018 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.075996 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wcpnl\"" Jul 22 02:34:42.076941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.076920 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 22 02:34:42.077015 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.076932 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 22 02:34:42.082893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.082869 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-rsp4q"] Jul 22 02:34:42.158756 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.158717 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9jpl\" (UniqueName: \"kubernetes.io/projected/0426d367-cac1-44df-a18d-a7176e5af680-kube-api-access-b9jpl\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.158932 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.158767 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0426d367-cac1-44df-a18d-a7176e5af680-webhook-cert\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.260045 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.260015 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9jpl\" (UniqueName: \"kubernetes.io/projected/0426d367-cac1-44df-a18d-a7176e5af680-kube-api-access-b9jpl\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.260293 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.260062 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0426d367-cac1-44df-a18d-a7176e5af680-webhook-cert\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.262670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.262646 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0426d367-cac1-44df-a18d-a7176e5af680-webhook-cert\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.269356 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.269330 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9jpl\" (UniqueName: \"kubernetes.io/projected/0426d367-cac1-44df-a18d-a7176e5af680-kube-api-access-b9jpl\") pod \"maas-controller-894c47d6f-rsp4q\" (UID: \"0426d367-cac1-44df-a18d-a7176e5af680\") " pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.385127 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.385033 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:42.521242 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.521216 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-894c47d6f-rsp4q"] Jul 22 02:34:42.523289 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:34:42.523263 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0426d367_cac1_44df_a18d_a7176e5af680.slice/crio-86e5dd78c7d1e457fd13a89d803520b2b9d5225795058eacb10af9d1274c19d3 WatchSource:0}: Error finding container 86e5dd78c7d1e457fd13a89d803520b2b9d5225795058eacb10af9d1274c19d3: Status 404 returned error can't find the container with id 86e5dd78c7d1e457fd13a89d803520b2b9d5225795058eacb10af9d1274c19d3 Jul 22 02:34:42.748030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:42.747985 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ee6349a8-a2c4-40a4-b813-0b8df5baa3ce" path="/var/lib/kubelet/pods/ee6349a8-a2c4-40a4-b813-0b8df5baa3ce/volumes" Jul 22 02:34:43.212966 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:43.212923 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-rsp4q" event={"ID":"0426d367-cac1-44df-a18d-a7176e5af680","Type":"ContainerStarted","Data":"84e92d38aac6c62dccc600c9ecb10c3d4174b7d1e010d221c9fedf26d7ea5cb3"} Jul 22 02:34:43.213463 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:43.212997 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:34:43.213463 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:43.213021 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-894c47d6f-rsp4q" event={"ID":"0426d367-cac1-44df-a18d-a7176e5af680","Type":"ContainerStarted","Data":"86e5dd78c7d1e457fd13a89d803520b2b9d5225795058eacb10af9d1274c19d3"} Jul 22 02:34:43.233441 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:43.233389 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-894c47d6f-rsp4q" podStartSLOduration=0.821892917 podStartE2EDuration="1.233370625s" totalImagesPullingTime="411.477708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:34:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:34:43.230215156 +0000 UTC m=+925.008291099" watchObservedRunningTime="2026-07-22 02:34:43.233370625 +0000 UTC m=+925.011446566" Jul 22 02:34:54.220612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:34:54.220581 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-894c47d6f-rsp4q" Jul 22 02:39:18.763854 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:39:18.763829 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:39:18.764433 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:39:18.763903 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:39:18.774987 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:39:18.774965 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:39:18.775175 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:39:18.775160 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:44:18.805447 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:44:18.805420 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:44:18.806017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:44:18.805688 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:44:18.814178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:44:18.814116 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:44:18.814288 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:44:18.814194 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:45:00.178883 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.178845 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq"] Jul 22 02:45:00.182275 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.182258 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" Jul 22 02:45:00.185434 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.185411 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:45:00.191831 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.191810 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" Jul 22 02:45:00.200360 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.200335 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq"] Jul 22 02:45:00.350920 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.350852 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq"] Jul 22 02:45:00.353322 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:45:00.353298 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ee59d10_e655_4ee7_8938_168c757496f3.slice/crio-a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81 WatchSource:0}: Error finding container a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81: Status 404 returned error can't find the container with id a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81 Jul 22 02:45:00.355314 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:00.355297 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:45:01.344928 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:01.344887 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" event={"ID":"9ee59d10-e655-4ee7-8938-168c757496f3","Type":"ContainerStarted","Data":"a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81"} Jul 22 02:45:04.368261 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:04.368220 2570 generic.go:356] "Generic (PLEG): container finished" podID="9ee59d10-e655-4ee7-8938-168c757496f3" containerID="b4c989451a9dc2046bfbba483de8fa5ca4d2ca73b001244a3c0da0542007ff7f" exitCode=0 Jul 22 02:45:04.368641 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:04.368266 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" event={"ID":"9ee59d10-e655-4ee7-8938-168c757496f3","Type":"ContainerDied","Data":"b4c989451a9dc2046bfbba483de8fa5ca4d2ca73b001244a3c0da0542007ff7f"} Jul 22 02:45:05.513381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:05.513354 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" Jul 22 02:45:06.380063 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:06.380037 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" Jul 22 02:45:06.380063 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:06.380050 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744805-pg7qq" event={"ID":"9ee59d10-e655-4ee7-8938-168c757496f3","Type":"ContainerDied","Data":"a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81"} Jul 22 02:45:06.380309 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:45:06.380098 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a243ad8831dec2b5445f0f116e95378ab8ebcce7ec86b067a20c5a56d3701b81" Jul 22 02:47:27.861795 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.861759 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:47:27.862368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.862099 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ee59d10-e655-4ee7-8938-168c757496f3" containerName="cleanup" Jul 22 02:47:27.862368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.862110 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ee59d10-e655-4ee7-8938-168c757496f3" containerName="cleanup" Jul 22 02:47:27.862368 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.862229 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ee59d10-e655-4ee7-8938-168c757496f3" containerName="cleanup" Jul 22 02:47:27.865276 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.865257 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:27.868162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.868146 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-f7e0def3-serving-cert\"" Jul 22 02:47:27.868251 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.868184 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:47:27.876108 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.876085 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:47:27.941910 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.941873 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:27.942088 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.941915 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgdrc\" (UniqueName: \"kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:27.942088 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:27.941998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.043001 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.042972 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgdrc\" (UniqueName: \"kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.043219 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.043032 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.043219 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.043073 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.043219 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:47:28.043188 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-serving-cert: secret "maas-api-e2e-ait-f7e0def3-serving-cert" not found Jul 22 02:47:28.043342 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:47:28.043239 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls podName:f8f36a0f-3661-4e97-a1a5-41d89ffdd76f nodeName:}" failed. No retries permitted until 2026-07-22 02:47:28.543224314 +0000 UTC m=+1690.321300233 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls") pod "maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" (UID: "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f") : secret "maas-api-e2e-ait-f7e0def3-serving-cert" not found Jul 22 02:47:28.043744 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.043723 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.073024 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.072986 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgdrc\" (UniqueName: \"kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.549569 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.549532 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.552278 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.552246 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") pod \"maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.776423 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.776390 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:28.920621 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:28.920594 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:47:28.922351 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:47:28.922327 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f36a0f_3661_4e97_a1a5_41d89ffdd76f.slice/crio-52d84fc1c96371906af64b2461b7658ae3cef9ceb003d3823e7801353b297c63 WatchSource:0}: Error finding container 52d84fc1c96371906af64b2461b7658ae3cef9ceb003d3823e7801353b297c63: Status 404 returned error can't find the container with id 52d84fc1c96371906af64b2461b7658ae3cef9ceb003d3823e7801353b297c63 Jul 22 02:47:29.118128 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:29.118048 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" event={"ID":"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f","Type":"ContainerStarted","Data":"52d84fc1c96371906af64b2461b7658ae3cef9ceb003d3823e7801353b297c63"} Jul 22 02:47:30.124752 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:30.124717 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" event={"ID":"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f","Type":"ContainerStarted","Data":"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31"} Jul 22 02:47:30.125182 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:30.124837 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:30.145637 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:30.145586 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" podStartSLOduration=2.62295079 podStartE2EDuration="3.145569276s" totalImagesPullingTime="522.618486ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:47:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:47:30.142657993 +0000 UTC m=+1691.920733933" watchObservedRunningTime="2026-07-22 02:47:30.145569276 +0000 UTC m=+1691.923645264" Jul 22 02:47:36.135628 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:36.135595 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:47:53.117791 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.117703 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862"] Jul 22 02:47:53.121432 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.121408 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:53.125163 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.125124 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:47:53.133884 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.133862 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862"] Jul 22 02:47:53.271936 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.271906 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7e0def3-ww862\" (UID: \"3868505d-c850-4748-b8ec-2a91f1a75c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:53.372807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.372712 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7e0def3-ww862\" (UID: \"3868505d-c850-4748-b8ec-2a91f1a75c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:53.373393 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.373372 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f7e0def3-ww862\" (UID: \"3868505d-c850-4748-b8ec-2a91f1a75c9c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:53.431707 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.431673 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:53.585623 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:53.585592 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862"] Jul 22 02:47:53.589174 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:47:53.589145 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3868505d_c850_4748_b8ec_2a91f1a75c9c.slice/crio-06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82 WatchSource:0}: Error finding container 06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82: Status 404 returned error can't find the container with id 06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82 Jul 22 02:47:54.245722 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:54.245687 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" event={"ID":"3868505d-c850-4748-b8ec-2a91f1a75c9c","Type":"ContainerStarted","Data":"91426b7f80e1ad4e539dc402c70dc86140b132f77c15ef082dd379894d760b8f"} Jul 22 02:47:54.246110 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:54.245746 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" event={"ID":"3868505d-c850-4748-b8ec-2a91f1a75c9c","Type":"ContainerStarted","Data":"06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82"} Jul 22 02:47:54.290361 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:54.290304 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" podStartSLOduration=1.290287124 podStartE2EDuration="1.290287124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:47:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:47:54.289206811 +0000 UTC m=+1716.067282753" watchObservedRunningTime="2026-07-22 02:47:54.290287124 +0000 UTC m=+1716.068363065" Jul 22 02:47:55.252251 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:55.252214 2570 generic.go:356] "Generic (PLEG): container finished" podID="3868505d-c850-4748-b8ec-2a91f1a75c9c" containerID="91426b7f80e1ad4e539dc402c70dc86140b132f77c15ef082dd379894d760b8f" exitCode=0 Jul 22 02:47:55.252635 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:55.252289 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" event={"ID":"3868505d-c850-4748-b8ec-2a91f1a75c9c","Type":"ContainerDied","Data":"91426b7f80e1ad4e539dc402c70dc86140b132f77c15ef082dd379894d760b8f"} Jul 22 02:47:56.390707 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:56.390680 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:56.501682 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:56.501645 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca\") pod \"3868505d-c850-4748-b8ec-2a91f1a75c9c\" (UID: \"3868505d-c850-4748-b8ec-2a91f1a75c9c\") " Jul 22 02:47:56.502012 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:56.501989 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3868505d-c850-4748-b8ec-2a91f1a75c9c" (UID: "3868505d-c850-4748-b8ec-2a91f1a75c9c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:47:56.603247 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:56.603110 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3868505d-c850-4748-b8ec-2a91f1a75c9c-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:47:57.263694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:57.263665 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" Jul 22 02:47:57.263875 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:57.263699 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862" event={"ID":"3868505d-c850-4748-b8ec-2a91f1a75c9c","Type":"ContainerDied","Data":"06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82"} Jul 22 02:47:57.263875 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:47:57.263751 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="06ef1cd91c2d9038b1a602905c18a449078cc1cbef403baf76ec15e704e9fa82" Jul 22 02:48:03.735894 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:03.735863 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:48:03.736451 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:03.736045 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" podUID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" containerName="maas-api" containerID="cri-o://531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31" gracePeriod=30 Jul 22 02:48:03.980073 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:03.980048 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:48:04.069511 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.069426 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgdrc\" (UniqueName: \"kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc\") pod \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " Jul 22 02:48:04.069511 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.069490 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca\") pod \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " Jul 22 02:48:04.069745 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.069533 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") pod \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\" (UID: \"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f\") " Jul 22 02:48:04.069860 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.069835 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" (UID: "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:48:04.071858 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.071836 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc" (OuterVolumeSpecName: "kube-api-access-xgdrc") pod "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" (UID: "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f"). InnerVolumeSpecName "kube-api-access-xgdrc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:04.071951 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.071907 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" (UID: "f8f36a0f-3661-4e97-a1a5-41d89ffdd76f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:48:04.170032 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.169997 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgdrc\" (UniqueName: \"kubernetes.io/projected/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-kube-api-access-xgdrc\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:04.170032 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.170026 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:04.170032 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.170035 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:04.301695 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.301664 2570 generic.go:356] "Generic (PLEG): container finished" podID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" containerID="531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31" exitCode=0 Jul 22 02:48:04.301898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.301719 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" Jul 22 02:48:04.301898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.301731 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" event={"ID":"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f","Type":"ContainerDied","Data":"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31"} Jul 22 02:48:04.301898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.301781 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9" event={"ID":"f8f36a0f-3661-4e97-a1a5-41d89ffdd76f","Type":"ContainerDied","Data":"52d84fc1c96371906af64b2461b7658ae3cef9ceb003d3823e7801353b297c63"} Jul 22 02:48:04.301898 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.301806 2570 scope.go:117] "RemoveContainer" containerID="531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31" Jul 22 02:48:04.316428 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.316407 2570 scope.go:117] "RemoveContainer" containerID="531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31" Jul 22 02:48:04.316725 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:04.316702 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31\": container with ID starting with 531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31 not found: ID does not exist" containerID="531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31" Jul 22 02:48:04.316776 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.316734 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31"} err="failed to get container status \"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31\": rpc error: code = NotFound desc = could not find container \"531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31\": container with ID starting with 531d2f3116ba9beeb9ee44006c24a2d2d964e1d30d1bd2c7a7b93b6b8690bb31 not found: ID does not exist" Jul 22 02:48:04.338701 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.338619 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:48:04.343044 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.343018 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-f7e0def3-76cd677dc5-cbzt9"] Jul 22 02:48:04.748687 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:04.748650 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" path="/var/lib/kubelet/pods/f8f36a0f-3661-4e97-a1a5-41d89ffdd76f/volumes" Jul 22 02:48:08.270064 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:08.270030 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862"] Jul 22 02:48:08.274772 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:08.274747 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f7e0def3-ww862"] Jul 22 02:48:08.748902 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:08.748858 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3868505d-c850-4748-b8ec-2a91f1a75c9c" path="/var/lib/kubelet/pods/3868505d-c850-4748-b8ec-2a91f1a75c9c/volumes" Jul 22 02:48:13.320743 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.320713 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:13.321229 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321212 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3868505d-c850-4748-b8ec-2a91f1a75c9c" containerName="revoke-keys" Jul 22 02:48:13.321308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321232 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3868505d-c850-4748-b8ec-2a91f1a75c9c" containerName="revoke-keys" Jul 22 02:48:13.321308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321278 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" containerName="maas-api" Jul 22 02:48:13.321308 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321287 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" containerName="maas-api" Jul 22 02:48:13.321455 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321374 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3868505d-c850-4748-b8ec-2a91f1a75c9c" containerName="revoke-keys" Jul 22 02:48:13.321455 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.321388 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="f8f36a0f-3661-4e97-a1a5-41d89ffdd76f" containerName="maas-api" Jul 22 02:48:13.325723 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.325703 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.333823 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.333802 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-d0d4ee22-serving-cert\"" Jul 22 02:48:13.333938 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.333805 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:48:13.349795 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.349760 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdcl2\" (UniqueName: \"kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.349972 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.349823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.349972 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.349917 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.380406 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.380371 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:13.450448 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.450416 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.450661 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.450479 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdcl2\" (UniqueName: \"kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.450661 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:13.450592 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-serving-cert: secret "maas-api-e2e-migrate-d0d4ee22-serving-cert" not found Jul 22 02:48:13.450661 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.450601 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.450809 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:13.450674 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls podName:17892fde-0961-4885-8505-93c56ffa8722 nodeName:}" failed. No retries permitted until 2026-07-22 02:48:13.950644055 +0000 UTC m=+1735.728719996 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls") pod "maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" (UID: "17892fde-0961-4885-8505-93c56ffa8722") : secret "maas-api-e2e-migrate-d0d4ee22-serving-cert" not found Jul 22 02:48:13.451252 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.451233 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.471749 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.471718 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdcl2\" (UniqueName: \"kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.955796 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.955745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:13.958294 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:13.958247 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") pod \"maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:14.237047 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:14.236957 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:14.389541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:14.389410 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:14.392360 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:48:14.392333 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17892fde_0961_4885_8505_93c56ffa8722.slice/crio-f7c45a07308f9a1bf9ad764c733e714894faab22c1b9d678261a5a24eac11024 WatchSource:0}: Error finding container f7c45a07308f9a1bf9ad764c733e714894faab22c1b9d678261a5a24eac11024: Status 404 returned error can't find the container with id f7c45a07308f9a1bf9ad764c733e714894faab22c1b9d678261a5a24eac11024 Jul 22 02:48:15.360619 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:15.360577 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" event={"ID":"17892fde-0961-4885-8505-93c56ffa8722","Type":"ContainerStarted","Data":"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2"} Jul 22 02:48:15.360804 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:15.360638 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" event={"ID":"17892fde-0961-4885-8505-93c56ffa8722","Type":"ContainerStarted","Data":"f7c45a07308f9a1bf9ad764c733e714894faab22c1b9d678261a5a24eac11024"} Jul 22 02:48:15.360804 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:15.360665 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:15.399827 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:15.399774 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" podStartSLOduration=2.032834408 podStartE2EDuration="2.399754724s" totalImagesPullingTime="366.920316ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:48:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:48:15.396797412 +0000 UTC m=+1737.174873345" watchObservedRunningTime="2026-07-22 02:48:15.399754724 +0000 UTC m=+1737.177830666" Jul 22 02:48:21.371801 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:21.371774 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:37.978752 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:37.978719 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d"] Jul 22 02:48:37.982331 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:37.982314 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:37.985440 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:37.985416 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:48:37.997387 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:37.997346 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d"] Jul 22 02:48:38.071056 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.071019 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d\" (UID: \"82f5340c-3b99-49d1-9ca5-7badaf256275\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:38.172527 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.172492 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d\" (UID: \"82f5340c-3b99-49d1-9ca5-7badaf256275\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:38.173082 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.173064 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d\" (UID: \"82f5340c-3b99-49d1-9ca5-7badaf256275\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:38.292435 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.292338 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:38.434859 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.434837 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d"] Jul 22 02:48:38.436480 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:48:38.436452 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f5340c_3b99_49d1_9ca5_7badaf256275.slice/crio-bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe WatchSource:0}: Error finding container bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe: Status 404 returned error can't find the container with id bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe Jul 22 02:48:38.484408 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:38.484379 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" event={"ID":"82f5340c-3b99-49d1-9ca5-7badaf256275","Type":"ContainerStarted","Data":"bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe"} Jul 22 02:48:39.494697 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:39.494662 2570 generic.go:356] "Generic (PLEG): container finished" podID="82f5340c-3b99-49d1-9ca5-7badaf256275" containerID="59e364c5bc5e219a645d28100f31aeae9810376d706a61bceb21641c779d8b77" exitCode=0 Jul 22 02:48:39.495318 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:39.495294 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" event={"ID":"82f5340c-3b99-49d1-9ca5-7badaf256275","Type":"ContainerDied","Data":"59e364c5bc5e219a645d28100f31aeae9810376d706a61bceb21641c779d8b77"} Jul 22 02:48:40.627684 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:40.627661 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:40.797728 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:40.797638 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca\") pod \"82f5340c-3b99-49d1-9ca5-7badaf256275\" (UID: \"82f5340c-3b99-49d1-9ca5-7badaf256275\") " Jul 22 02:48:40.798000 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:40.797978 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "82f5340c-3b99-49d1-9ca5-7badaf256275" (UID: "82f5340c-3b99-49d1-9ca5-7badaf256275"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:48:40.898692 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:40.898653 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/82f5340c-3b99-49d1-9ca5-7badaf256275-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:41.506664 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:41.506630 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" Jul 22 02:48:41.506843 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:41.506667 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d" event={"ID":"82f5340c-3b99-49d1-9ca5-7badaf256275","Type":"ContainerDied","Data":"bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe"} Jul 22 02:48:41.506843 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:41.506712 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bc15a21169a2f666a779b5beca8c3ea0a66293e731628ee6fd18fbde9a696fbe" Jul 22 02:48:48.873432 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:48.873402 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:48.873793 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:48.873620 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" podUID="17892fde-0961-4885-8505-93c56ffa8722" containerName="maas-api" containerID="cri-o://597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2" gracePeriod=30 Jul 22 02:48:49.142029 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.142006 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:49.174386 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.174353 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca\") pod \"17892fde-0961-4885-8505-93c56ffa8722\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " Jul 22 02:48:49.174386 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.174393 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sdcl2\" (UniqueName: \"kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2\") pod \"17892fde-0961-4885-8505-93c56ffa8722\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " Jul 22 02:48:49.174566 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.174438 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") pod \"17892fde-0961-4885-8505-93c56ffa8722\" (UID: \"17892fde-0961-4885-8505-93c56ffa8722\") " Jul 22 02:48:49.174756 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.174732 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17892fde-0961-4885-8505-93c56ffa8722" (UID: "17892fde-0961-4885-8505-93c56ffa8722"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:48:49.176758 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.176728 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2" (OuterVolumeSpecName: "kube-api-access-sdcl2") pod "17892fde-0961-4885-8505-93c56ffa8722" (UID: "17892fde-0961-4885-8505-93c56ffa8722"). InnerVolumeSpecName "kube-api-access-sdcl2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:48:49.176758 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.176743 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17892fde-0961-4885-8505-93c56ffa8722" (UID: "17892fde-0961-4885-8505-93c56ffa8722"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:48:49.275519 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.275475 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17892fde-0961-4885-8505-93c56ffa8722-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:49.275519 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.275513 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sdcl2\" (UniqueName: \"kubernetes.io/projected/17892fde-0961-4885-8505-93c56ffa8722-kube-api-access-sdcl2\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:49.275519 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.275524 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17892fde-0961-4885-8505-93c56ffa8722-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:48:49.548403 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.548371 2570 generic.go:356] "Generic (PLEG): container finished" podID="17892fde-0961-4885-8505-93c56ffa8722" containerID="597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2" exitCode=0 Jul 22 02:48:49.548607 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.548419 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" Jul 22 02:48:49.548607 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.548455 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" event={"ID":"17892fde-0961-4885-8505-93c56ffa8722","Type":"ContainerDied","Data":"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2"} Jul 22 02:48:49.548607 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.548517 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln" event={"ID":"17892fde-0961-4885-8505-93c56ffa8722","Type":"ContainerDied","Data":"f7c45a07308f9a1bf9ad764c733e714894faab22c1b9d678261a5a24eac11024"} Jul 22 02:48:49.548607 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.548546 2570 scope.go:117] "RemoveContainer" containerID="597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2" Jul 22 02:48:49.558603 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.558584 2570 scope.go:117] "RemoveContainer" containerID="597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2" Jul 22 02:48:49.558895 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:49.558875 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2\": container with ID starting with 597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2 not found: ID does not exist" containerID="597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2" Jul 22 02:48:49.558957 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.558905 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2"} err="failed to get container status \"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2\": rpc error: code = NotFound desc = could not find container \"597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2\": container with ID starting with 597a44dd21159a792ce645d92b8a90137b69a4e4f0e9972ff2bc24400c5c53a2 not found: ID does not exist" Jul 22 02:48:49.580503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.580473 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:49.593953 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:49.593925 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-d0d4ee22-64b578d9f4-hw7ln"] Jul 22 02:48:50.749083 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:50.749045 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17892fde-0961-4885-8505-93c56ffa8722" path="/var/lib/kubelet/pods/17892fde-0961-4885-8505-93c56ffa8722/volumes" Jul 22 02:48:53.269310 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:53.269274 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d"] Jul 22 02:48:53.275328 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:53.275300 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-d0d4ee22-gfk5d"] Jul 22 02:48:54.748705 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:54.748672 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82f5340c-3b99-49d1-9ca5-7badaf256275" path="/var/lib/kubelet/pods/82f5340c-3b99-49d1-9ca5-7badaf256275/volumes" Jul 22 02:48:57.591455 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591428 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:48:57.591816 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591757 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82f5340c-3b99-49d1-9ca5-7badaf256275" containerName="revoke-keys" Jul 22 02:48:57.591816 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591769 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="82f5340c-3b99-49d1-9ca5-7badaf256275" containerName="revoke-keys" Jul 22 02:48:57.591816 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591784 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17892fde-0961-4885-8505-93c56ffa8722" containerName="maas-api" Jul 22 02:48:57.591816 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591789 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="17892fde-0961-4885-8505-93c56ffa8722" containerName="maas-api" Jul 22 02:48:57.591945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591851 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="17892fde-0961-4885-8505-93c56ffa8722" containerName="maas-api" Jul 22 02:48:57.591945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.591858 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="82f5340c-3b99-49d1-9ca5-7badaf256275" containerName="revoke-keys" Jul 22 02:48:57.596197 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.596174 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.607983 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:57.607955 2570 status_manager.go:1018] "Failed to get status for pod" err="pods \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" is forbidden: User \"system:node:ip-10-0-142-149.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-149.ec2.internal' and this object" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.607983 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:57.607960 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-e2e-ait-7833c972-serving-cert\" is forbidden: User \"system:node:ip-10-0-142-149.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-149.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7833c972-serving-cert\"" type="*v1.Secret" Jul 22 02:48:57.607983 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:57.607960 2570 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"maas-api-dockercfg-q9rm7\" is forbidden: User \"system:node:ip-10-0-142-149.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"odh-ai-gateway-infra\": no relationship found between node 'ip-10-0-142-149.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" type="*v1.Secret" Jul 22 02:48:57.638702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.638672 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.638702 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.638704 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.638917 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.638811 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hbd7x\" (UniqueName: \"kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.700460 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.700414 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:48:57.739790 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.739755 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hbd7x\" (UniqueName: \"kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.739962 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.739818 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.739962 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.739837 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.740421 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.740405 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:57.781851 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:57.781816 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hbd7x\" (UniqueName: \"kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:58.740874 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:58.740839 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-serving-cert: failed to sync secret cache: timed out waiting for the condition Jul 22 02:48:58.741287 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:48:58.740924 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls podName:51e037e4-a886-47ac-a045-60b0818c2e6e nodeName:}" failed. No retries permitted until 2026-07-22 02:48:59.240904089 +0000 UTC m=+1781.018980028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls") pod "maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" (UID: "51e037e4-a886-47ac-a045-60b0818c2e6e") : failed to sync secret cache: timed out waiting for the condition Jul 22 02:48:58.958387 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:58.958353 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-7833c972-serving-cert\"" Jul 22 02:48:59.081895 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:59.081810 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:48:59.254272 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:59.254229 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:59.256769 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:59.256740 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") pod \"maas-api-e2e-ait-7833c972-fd977f67c-jbqkk\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:59.405628 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:59.405544 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:48:59.621917 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:48:59.621892 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51e037e4_a886_47ac_a045_60b0818c2e6e.slice/crio-dd4ed67c747591848ae3e2dc8aa160e744fe7072657a8751d215ca38d0dba599 WatchSource:0}: Error finding container dd4ed67c747591848ae3e2dc8aa160e744fe7072657a8751d215ca38d0dba599: Status 404 returned error can't find the container with id dd4ed67c747591848ae3e2dc8aa160e744fe7072657a8751d215ca38d0dba599 Jul 22 02:48:59.631956 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:48:59.631932 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:49:00.609097 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:00.609063 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" event={"ID":"51e037e4-a886-47ac-a045-60b0818c2e6e","Type":"ContainerStarted","Data":"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924"} Jul 22 02:49:00.609522 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:00.609118 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" event={"ID":"51e037e4-a886-47ac-a045-60b0818c2e6e","Type":"ContainerStarted","Data":"dd4ed67c747591848ae3e2dc8aa160e744fe7072657a8751d215ca38d0dba599"} Jul 22 02:49:00.609522 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:00.609194 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:49:00.657618 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:00.657566 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" podStartSLOduration=3.215699741 podStartE2EDuration="3.657551174s" totalImagesPullingTime="441.851433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:48:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:49:00.657443181 +0000 UTC m=+1782.435519124" watchObservedRunningTime="2026-07-22 02:49:00.657551174 +0000 UTC m=+1782.435627113" Jul 22 02:49:06.620039 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:06.620004 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:49:18.839728 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:18.839696 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:49:18.842158 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:18.842120 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:49:18.856401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:18.856371 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:49:18.858610 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:18.858590 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:49:23.022862 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.022824 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54"] Jul 22 02:49:23.026431 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.026407 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:23.029872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.029853 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:49:23.037330 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.037305 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54"] Jul 22 02:49:23.177209 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.177172 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7833c972-d2g54\" (UID: \"b74edbad-e986-484f-adfb-6ce8fc5a44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:23.277974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.277886 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7833c972-d2g54\" (UID: \"b74edbad-e986-484f-adfb-6ce8fc5a44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:23.278646 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.278624 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7833c972-d2g54\" (UID: \"b74edbad-e986-484f-adfb-6ce8fc5a44f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:23.337684 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.337647 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:23.478365 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.478331 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54"] Jul 22 02:49:23.480580 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:49:23.480555 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb74edbad_e986_484f_adfb_6ce8fc5a44f1.slice/crio-99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123 WatchSource:0}: Error finding container 99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123: Status 404 returned error can't find the container with id 99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123 Jul 22 02:49:23.740222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:23.740183 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" event={"ID":"b74edbad-e986-484f-adfb-6ce8fc5a44f1","Type":"ContainerStarted","Data":"99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123"} Jul 22 02:49:24.747060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:24.747027 2570 generic.go:356] "Generic (PLEG): container finished" podID="b74edbad-e986-484f-adfb-6ce8fc5a44f1" containerID="cda7ec2a39bed6ee3740d9af7d1dd79d3fde9b744b348206d01b5ab9885c9f8f" exitCode=0 Jul 22 02:49:24.747963 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:24.747938 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" event={"ID":"b74edbad-e986-484f-adfb-6ce8fc5a44f1","Type":"ContainerDied","Data":"cda7ec2a39bed6ee3740d9af7d1dd79d3fde9b744b348206d01b5ab9885c9f8f"} Jul 22 02:49:25.896553 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:25.896525 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:26.001647 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.001610 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca\") pod \"b74edbad-e986-484f-adfb-6ce8fc5a44f1\" (UID: \"b74edbad-e986-484f-adfb-6ce8fc5a44f1\") " Jul 22 02:49:26.001974 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.001950 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b74edbad-e986-484f-adfb-6ce8fc5a44f1" (UID: "b74edbad-e986-484f-adfb-6ce8fc5a44f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:49:26.102971 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.102893 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b74edbad-e986-484f-adfb-6ce8fc5a44f1-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:49:26.760200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.760178 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" Jul 22 02:49:26.760337 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.760176 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54" event={"ID":"b74edbad-e986-484f-adfb-6ce8fc5a44f1","Type":"ContainerDied","Data":"99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123"} Jul 22 02:49:26.760337 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:26.760293 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="99298a9af720b65a72af233d4686067ed1700b6471c15e929018870acb0a6123" Jul 22 02:49:33.448522 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.448488 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:49:33.449011 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.448704 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" containerName="maas-api" containerID="cri-o://136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924" gracePeriod=30 Jul 22 02:49:33.697265 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.697241 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:49:33.797290 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.797253 2570 generic.go:356] "Generic (PLEG): container finished" podID="51e037e4-a886-47ac-a045-60b0818c2e6e" containerID="136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924" exitCode=0 Jul 22 02:49:33.797480 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.797304 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" Jul 22 02:49:33.797480 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.797342 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" event={"ID":"51e037e4-a886-47ac-a045-60b0818c2e6e","Type":"ContainerDied","Data":"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924"} Jul 22 02:49:33.797480 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.797396 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk" event={"ID":"51e037e4-a886-47ac-a045-60b0818c2e6e","Type":"ContainerDied","Data":"dd4ed67c747591848ae3e2dc8aa160e744fe7072657a8751d215ca38d0dba599"} Jul 22 02:49:33.797480 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.797412 2570 scope.go:117] "RemoveContainer" containerID="136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924" Jul 22 02:49:33.806549 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.806526 2570 scope.go:117] "RemoveContainer" containerID="136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924" Jul 22 02:49:33.806816 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:49:33.806793 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924\": container with ID starting with 136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924 not found: ID does not exist" containerID="136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924" Jul 22 02:49:33.806862 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.806828 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924"} err="failed to get container status \"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924\": rpc error: code = NotFound desc = could not find container \"136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924\": container with ID starting with 136d96275a2b10f57d020d0acd2e048748ed7629ffe47ff72c9a8826c53de924 not found: ID does not exist" Jul 22 02:49:33.874287 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.874255 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") pod \"51e037e4-a886-47ac-a045-60b0818c2e6e\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " Jul 22 02:49:33.874476 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.874395 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca\") pod \"51e037e4-a886-47ac-a045-60b0818c2e6e\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " Jul 22 02:49:33.874476 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.874439 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hbd7x\" (UniqueName: \"kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x\") pod \"51e037e4-a886-47ac-a045-60b0818c2e6e\" (UID: \"51e037e4-a886-47ac-a045-60b0818c2e6e\") " Jul 22 02:49:33.874755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.874732 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "51e037e4-a886-47ac-a045-60b0818c2e6e" (UID: "51e037e4-a886-47ac-a045-60b0818c2e6e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:49:33.876624 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.876602 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "51e037e4-a886-47ac-a045-60b0818c2e6e" (UID: "51e037e4-a886-47ac-a045-60b0818c2e6e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:49:33.876710 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.876625 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x" (OuterVolumeSpecName: "kube-api-access-hbd7x") pod "51e037e4-a886-47ac-a045-60b0818c2e6e" (UID: "51e037e4-a886-47ac-a045-60b0818c2e6e"). InnerVolumeSpecName "kube-api-access-hbd7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:49:33.975540 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.975491 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/51e037e4-a886-47ac-a045-60b0818c2e6e-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:49:33.975540 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.975536 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hbd7x\" (UniqueName: \"kubernetes.io/projected/51e037e4-a886-47ac-a045-60b0818c2e6e-kube-api-access-hbd7x\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:49:33.975540 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:33.975546 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/51e037e4-a886-47ac-a045-60b0818c2e6e-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:49:34.120904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:34.120868 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:49:34.125378 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:34.125355 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-7833c972-fd977f67c-jbqkk"] Jul 22 02:49:34.749041 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:34.749008 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" path="/var/lib/kubelet/pods/51e037e4-a886-47ac-a045-60b0818c2e6e/volumes" Jul 22 02:49:38.164180 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:38.164145 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54"] Jul 22 02:49:38.169687 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:38.169653 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7833c972-d2g54"] Jul 22 02:49:38.750699 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:38.750666 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b74edbad-e986-484f-adfb-6ce8fc5a44f1" path="/var/lib/kubelet/pods/b74edbad-e986-484f-adfb-6ce8fc5a44f1/volumes" Jul 22 02:49:42.588197 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588164 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:49:42.588570 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588557 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" containerName="maas-api" Jul 22 02:49:42.588618 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588573 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" containerName="maas-api" Jul 22 02:49:42.588618 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588585 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b74edbad-e986-484f-adfb-6ce8fc5a44f1" containerName="revoke-keys" Jul 22 02:49:42.588618 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588590 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b74edbad-e986-484f-adfb-6ce8fc5a44f1" containerName="revoke-keys" Jul 22 02:49:42.588711 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588662 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="51e037e4-a886-47ac-a045-60b0818c2e6e" containerName="maas-api" Jul 22 02:49:42.588711 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.588674 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b74edbad-e986-484f-adfb-6ce8fc5a44f1" containerName="revoke-keys" Jul 22 02:49:42.593077 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.593058 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.595880 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.595859 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-54774162-serving-cert\"" Jul 22 02:49:42.597706 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.597693 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:49:42.610917 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.610889 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:49:42.649983 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.649953 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h479w\" (UniqueName: \"kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.650214 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.650008 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.650214 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.650126 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.751520 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.751464 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.751709 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.751566 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.751709 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.751620 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h479w\" (UniqueName: \"kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.751838 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:49:42.751714 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-derive-54774162-serving-cert: secret "maas-api-e2e-derive-54774162-serving-cert" not found Jul 22 02:49:42.751838 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:49:42.751795 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls podName:df84f2b9-8204-4559-adc0-b6c58001b113 nodeName:}" failed. No retries permitted until 2026-07-22 02:49:43.25177188 +0000 UTC m=+1825.029847808 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls") pod "maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" (UID: "df84f2b9-8204-4559-adc0-b6c58001b113") : secret "maas-api-e2e-derive-54774162-serving-cert" not found Jul 22 02:49:42.752225 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.752205 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:42.761759 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:42.761727 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h479w\" (UniqueName: \"kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:43.255943 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:43.255907 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:43.258608 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:43.258579 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") pod \"maas-api-e2e-derive-54774162-5fdff6676b-xz2hm\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:43.503447 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:43.503412 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:43.662990 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:43.662964 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:49:43.665737 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:49:43.665711 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddf84f2b9_8204_4559_adc0_b6c58001b113.slice/crio-86fa97a82dfe7b93354212e255b340c0b6c960d34d1b7e1e7a8675e6e77fad0a WatchSource:0}: Error finding container 86fa97a82dfe7b93354212e255b340c0b6c960d34d1b7e1e7a8675e6e77fad0a: Status 404 returned error can't find the container with id 86fa97a82dfe7b93354212e255b340c0b6c960d34d1b7e1e7a8675e6e77fad0a Jul 22 02:49:43.856870 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:43.856782 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" event={"ID":"df84f2b9-8204-4559-adc0-b6c58001b113","Type":"ContainerStarted","Data":"86fa97a82dfe7b93354212e255b340c0b6c960d34d1b7e1e7a8675e6e77fad0a"} Jul 22 02:49:44.863813 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:44.863777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" event={"ID":"df84f2b9-8204-4559-adc0-b6c58001b113","Type":"ContainerStarted","Data":"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04"} Jul 22 02:49:44.864205 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:44.863897 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:49:44.886126 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:44.886075 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" podStartSLOduration=2.5303923470000003 podStartE2EDuration="2.886056984s" totalImagesPullingTime="355.664637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:49:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:49:44.883220192 +0000 UTC m=+1826.661296130" watchObservedRunningTime="2026-07-22 02:49:44.886056984 +0000 UTC m=+1826.664132923" Jul 22 02:49:50.875012 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:49:50.874983 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:50:07.199123 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.199079 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf"] Jul 22 02:50:07.202458 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.202440 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:07.205473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.205452 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:50:07.216465 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.216429 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf"] Jul 22 02:50:07.268229 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.268191 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-54774162-bvnxf\" (UID: \"5af10495-dae7-482f-9c71-dfc026548661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:07.369210 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.369177 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-54774162-bvnxf\" (UID: \"5af10495-dae7-482f-9c71-dfc026548661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:07.369808 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.369790 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-54774162-bvnxf\" (UID: \"5af10495-dae7-482f-9c71-dfc026548661\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:07.516049 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.516011 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:07.659604 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.659543 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf"] Jul 22 02:50:07.661918 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:50:07.661893 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5af10495_dae7_482f_9c71_dfc026548661.slice/crio-c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf WatchSource:0}: Error finding container c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf: Status 404 returned error can't find the container with id c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf Jul 22 02:50:07.993685 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.993632 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" event={"ID":"5af10495-dae7-482f-9c71-dfc026548661","Type":"ContainerStarted","Data":"8de1336b8067163efeee82a45783dc5c32b43999d4b4428a10ecfc74e96848dd"} Jul 22 02:50:07.993873 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:07.993731 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" event={"ID":"5af10495-dae7-482f-9c71-dfc026548661","Type":"ContainerStarted","Data":"c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf"} Jul 22 02:50:08.011618 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:08.011564 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" podStartSLOduration=1.011546775 podStartE2EDuration="1.011546775s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:50:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:50:08.008914233 +0000 UTC m=+1849.786990173" watchObservedRunningTime="2026-07-22 02:50:08.011546775 +0000 UTC m=+1849.789622718" Jul 22 02:50:09.001162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:09.001050 2570 generic.go:356] "Generic (PLEG): container finished" podID="5af10495-dae7-482f-9c71-dfc026548661" containerID="8de1336b8067163efeee82a45783dc5c32b43999d4b4428a10ecfc74e96848dd" exitCode=0 Jul 22 02:50:09.001162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:09.001101 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" event={"ID":"5af10495-dae7-482f-9c71-dfc026548661","Type":"ContainerDied","Data":"8de1336b8067163efeee82a45783dc5c32b43999d4b4428a10ecfc74e96848dd"} Jul 22 02:50:10.158741 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:10.158714 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:10.296379 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:10.296279 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca\") pod \"5af10495-dae7-482f-9c71-dfc026548661\" (UID: \"5af10495-dae7-482f-9c71-dfc026548661\") " Jul 22 02:50:10.296682 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:10.296657 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "5af10495-dae7-482f-9c71-dfc026548661" (UID: "5af10495-dae7-482f-9c71-dfc026548661"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:50:10.397077 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:10.397043 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/5af10495-dae7-482f-9c71-dfc026548661-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:50:11.013637 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:11.013543 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" Jul 22 02:50:11.013637 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:11.013585 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf" event={"ID":"5af10495-dae7-482f-9c71-dfc026548661","Type":"ContainerDied","Data":"c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf"} Jul 22 02:50:11.013637 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:11.013635 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c5b51ced6c99e22bf171b3278dfacc9321b11d40275b7bfa888aec7cda2085bf" Jul 22 02:50:17.623152 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.623095 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:50:17.623617 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.623404 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" podUID="df84f2b9-8204-4559-adc0-b6c58001b113" containerName="maas-api" containerID="cri-o://794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04" gracePeriod=30 Jul 22 02:50:17.872295 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.872268 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:50:17.964731 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.964694 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca\") pod \"df84f2b9-8204-4559-adc0-b6c58001b113\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " Jul 22 02:50:17.964896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.964743 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h479w\" (UniqueName: \"kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w\") pod \"df84f2b9-8204-4559-adc0-b6c58001b113\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " Jul 22 02:50:17.964896 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.964765 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") pod \"df84f2b9-8204-4559-adc0-b6c58001b113\" (UID: \"df84f2b9-8204-4559-adc0-b6c58001b113\") " Jul 22 02:50:17.965076 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.965055 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "df84f2b9-8204-4559-adc0-b6c58001b113" (UID: "df84f2b9-8204-4559-adc0-b6c58001b113"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:50:17.967037 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.967004 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "df84f2b9-8204-4559-adc0-b6c58001b113" (UID: "df84f2b9-8204-4559-adc0-b6c58001b113"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:50:17.967186 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:17.967051 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w" (OuterVolumeSpecName: "kube-api-access-h479w") pod "df84f2b9-8204-4559-adc0-b6c58001b113" (UID: "df84f2b9-8204-4559-adc0-b6c58001b113"). InnerVolumeSpecName "kube-api-access-h479w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:50:18.052046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.052015 2570 generic.go:356] "Generic (PLEG): container finished" podID="df84f2b9-8204-4559-adc0-b6c58001b113" containerID="794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04" exitCode=0 Jul 22 02:50:18.052238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.052061 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" Jul 22 02:50:18.052238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.052101 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" event={"ID":"df84f2b9-8204-4559-adc0-b6c58001b113","Type":"ContainerDied","Data":"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04"} Jul 22 02:50:18.052238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.052176 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm" event={"ID":"df84f2b9-8204-4559-adc0-b6c58001b113","Type":"ContainerDied","Data":"86fa97a82dfe7b93354212e255b340c0b6c960d34d1b7e1e7a8675e6e77fad0a"} Jul 22 02:50:18.052238 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.052199 2570 scope.go:117] "RemoveContainer" containerID="794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04" Jul 22 02:50:18.061886 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.061867 2570 scope.go:117] "RemoveContainer" containerID="794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04" Jul 22 02:50:18.062146 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:50:18.062116 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04\": container with ID starting with 794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04 not found: ID does not exist" containerID="794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04" Jul 22 02:50:18.062216 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.062157 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04"} err="failed to get container status \"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04\": rpc error: code = NotFound desc = could not find container \"794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04\": container with ID starting with 794d7f0434ebd294b89f929352971db292f427e963e6f413a569ec44d46c8f04 not found: ID does not exist" Jul 22 02:50:18.065691 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.065667 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/df84f2b9-8204-4559-adc0-b6c58001b113-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:50:18.065826 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.065806 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/df84f2b9-8204-4559-adc0-b6c58001b113-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:50:18.065894 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.065831 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h479w\" (UniqueName: \"kubernetes.io/projected/df84f2b9-8204-4559-adc0-b6c58001b113-kube-api-access-h479w\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:50:18.079510 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.079468 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:50:18.082838 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.082813 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-54774162-5fdff6676b-xz2hm"] Jul 22 02:50:18.748868 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:18.748838 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df84f2b9-8204-4559-adc0-b6c58001b113" path="/var/lib/kubelet/pods/df84f2b9-8204-4559-adc0-b6c58001b113/volumes" Jul 22 02:50:22.265342 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:22.265297 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf"] Jul 22 02:50:22.268212 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:22.268188 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-54774162-bvnxf"] Jul 22 02:50:22.748147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:50:22.748091 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5af10495-dae7-482f-9c71-dfc026548661" path="/var/lib/kubelet/pods/5af10495-dae7-482f-9c71-dfc026548661/volumes" Jul 22 02:52:06.765222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765191 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:06.765810 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765712 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df84f2b9-8204-4559-adc0-b6c58001b113" containerName="maas-api" Jul 22 02:52:06.765810 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765732 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="df84f2b9-8204-4559-adc0-b6c58001b113" containerName="maas-api" Jul 22 02:52:06.765810 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765750 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5af10495-dae7-482f-9c71-dfc026548661" containerName="revoke-keys" Jul 22 02:52:06.765810 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765759 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5af10495-dae7-482f-9c71-dfc026548661" containerName="revoke-keys" Jul 22 02:52:06.766022 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765837 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="df84f2b9-8204-4559-adc0-b6c58001b113" containerName="maas-api" Jul 22 02:52:06.766022 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.765849 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="5af10495-dae7-482f-9c71-dfc026548661" containerName="revoke-keys" Jul 22 02:52:06.769124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.769101 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.772338 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.772308 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:52:06.772477 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.772350 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-4151b7b4-serving-cert\"" Jul 22 02:52:06.789672 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.789641 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:06.890475 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.890442 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lrhdb\" (UniqueName: \"kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.890655 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.890500 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.890655 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.890564 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.991596 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.991560 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.991759 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.991605 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.991759 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.991674 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lrhdb\" (UniqueName: \"kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:06.991759 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:52:06.991737 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-serving-cert: secret "maas-api-e2e-mt-4151b7b4-serving-cert" not found Jul 22 02:52:06.991940 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:52:06.991810 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls podName:16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e nodeName:}" failed. No retries permitted until 2026-07-22 02:52:07.491788937 +0000 UTC m=+1969.269864855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls") pod "maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" (UID: "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e") : secret "maas-api-e2e-mt-4151b7b4-serving-cert" not found Jul 22 02:52:06.992357 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:06.992336 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:07.007373 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.007341 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lrhdb\" (UniqueName: \"kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:07.496826 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.496784 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:07.499619 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.499590 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") pod \"maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:07.679597 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.679564 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:07.830829 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.830776 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:07.833089 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:52:07.833057 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16ad6a8e_3ee6_4dba_8e2e_31d4ef51043e.slice/crio-9e4b564569d605e1bc6585b851f6b118341f5534131739348975d1ecb8656dcf WatchSource:0}: Error finding container 9e4b564569d605e1bc6585b851f6b118341f5534131739348975d1ecb8656dcf: Status 404 returned error can't find the container with id 9e4b564569d605e1bc6585b851f6b118341f5534131739348975d1ecb8656dcf Jul 22 02:52:07.834524 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:07.834507 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:52:08.652888 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:08.652794 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" event={"ID":"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e","Type":"ContainerStarted","Data":"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a"} Jul 22 02:52:08.652888 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:08.652849 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" event={"ID":"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e","Type":"ContainerStarted","Data":"9e4b564569d605e1bc6585b851f6b118341f5534131739348975d1ecb8656dcf"} Jul 22 02:52:08.653102 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:08.652941 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:08.692365 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:08.692308 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" podStartSLOduration=2.159656653 podStartE2EDuration="2.692291623s" totalImagesPullingTime="532.63497ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:52:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:52:08.689197635 +0000 UTC m=+1970.467273575" watchObservedRunningTime="2026-07-22 02:52:08.692291623 +0000 UTC m=+1970.470367563" Jul 22 02:52:14.663488 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:14.663460 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:18.427507 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.427473 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt"] Jul 22 02:52:18.431302 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.431283 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:18.435050 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.435024 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:52:18.441749 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.441725 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt"] Jul 22 02:52:18.498017 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.497974 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt\" (UID: \"11e0aac9-5ea3-4ebc-bd38-37761997e5be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:18.599019 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.598977 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt\" (UID: \"11e0aac9-5ea3-4ebc-bd38-37761997e5be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:18.599684 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.599664 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt\" (UID: \"11e0aac9-5ea3-4ebc-bd38-37761997e5be\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:18.745061 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.745025 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:52:18.752632 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.752607 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:18.896547 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:18.896510 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt"] Jul 22 02:52:18.899485 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:52:18.899455 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11e0aac9_5ea3_4ebc_bd38_37761997e5be.slice/crio-d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885 WatchSource:0}: Error finding container d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885: Status 404 returned error can't find the container with id d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885 Jul 22 02:52:19.720582 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:19.720543 2570 generic.go:356] "Generic (PLEG): container finished" podID="11e0aac9-5ea3-4ebc-bd38-37761997e5be" containerID="83cfb146c82db2fe8b0c9419b16c4b16468748da5031470ff560160ea0c82a86" exitCode=0 Jul 22 02:52:19.720582 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:19.720586 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" event={"ID":"11e0aac9-5ea3-4ebc-bd38-37761997e5be","Type":"ContainerDied","Data":"83cfb146c82db2fe8b0c9419b16c4b16468748da5031470ff560160ea0c82a86"} Jul 22 02:52:19.721242 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:19.720629 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" event={"ID":"11e0aac9-5ea3-4ebc-bd38-37761997e5be","Type":"ContainerStarted","Data":"d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885"} Jul 22 02:52:20.866650 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:20.866625 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:21.022496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.022408 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca\") pod \"11e0aac9-5ea3-4ebc-bd38-37761997e5be\" (UID: \"11e0aac9-5ea3-4ebc-bd38-37761997e5be\") " Jul 22 02:52:21.022805 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.022783 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "11e0aac9-5ea3-4ebc-bd38-37761997e5be" (UID: "11e0aac9-5ea3-4ebc-bd38-37761997e5be"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:52:21.123636 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.123603 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/11e0aac9-5ea3-4ebc-bd38-37761997e5be-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:52:21.732593 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.732564 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" Jul 22 02:52:21.732593 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.732598 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt" event={"ID":"11e0aac9-5ea3-4ebc-bd38-37761997e5be","Type":"ContainerDied","Data":"d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885"} Jul 22 02:52:21.732814 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:21.732643 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d1b880c8b05cb0b1c188d105e01c046cb5f4d2c88b1e14e7e86b13f7a90d9885" Jul 22 02:52:29.030895 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.030863 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:29.031327 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.031109 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" podUID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" containerName="maas-api" containerID="cri-o://ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a" gracePeriod=30 Jul 22 02:52:29.287941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.287881 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:29.396662 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.396625 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") pod \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " Jul 22 02:52:29.396835 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.396675 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca\") pod \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " Jul 22 02:52:29.396835 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.396703 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lrhdb\" (UniqueName: \"kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb\") pod \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\" (UID: \"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e\") " Jul 22 02:52:29.397112 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.397072 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" (UID: "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:52:29.398958 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.398936 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb" (OuterVolumeSpecName: "kube-api-access-lrhdb") pod "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" (UID: "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e"). InnerVolumeSpecName "kube-api-access-lrhdb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:52:29.399083 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.398959 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" (UID: "16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:52:29.498178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.498101 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:52:29.498178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.498174 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:52:29.498178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.498184 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lrhdb\" (UniqueName: \"kubernetes.io/projected/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e-kube-api-access-lrhdb\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:52:29.784729 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.784692 2570 generic.go:356] "Generic (PLEG): container finished" podID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" containerID="ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a" exitCode=0 Jul 22 02:52:29.784945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.784737 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" Jul 22 02:52:29.784945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.784742 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" event={"ID":"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e","Type":"ContainerDied","Data":"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a"} Jul 22 02:52:29.784945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.784793 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd" event={"ID":"16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e","Type":"ContainerDied","Data":"9e4b564569d605e1bc6585b851f6b118341f5534131739348975d1ecb8656dcf"} Jul 22 02:52:29.784945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.784822 2570 scope.go:117] "RemoveContainer" containerID="ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a" Jul 22 02:52:29.794763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.794738 2570 scope.go:117] "RemoveContainer" containerID="ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a" Jul 22 02:52:29.795054 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:52:29.795033 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a\": container with ID starting with ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a not found: ID does not exist" containerID="ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a" Jul 22 02:52:29.795147 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.795065 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a"} err="failed to get container status \"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a\": rpc error: code = NotFound desc = could not find container \"ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a\": container with ID starting with ccd9bf92dce0e83d506a4548e32429233677719a5872f1f792cd986a17f2760a not found: ID does not exist" Jul 22 02:52:29.808089 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.808061 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:29.811611 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:29.811582 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-4151b7b4-56b7d9778d-q6tdd"] Jul 22 02:52:30.749108 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:30.749074 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" path="/var/lib/kubelet/pods/16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e/volumes" Jul 22 02:52:33.500982 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:33.500948 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt"] Jul 22 02:52:33.504000 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:33.503977 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-4151b7b4-6fzdt"] Jul 22 02:52:34.748999 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:52:34.748966 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="11e0aac9-5ea3-4ebc-bd38-37761997e5be" path="/var/lib/kubelet/pods/11e0aac9-5ea3-4ebc-bd38-37761997e5be/volumes" Jul 22 02:53:04.059436 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059401 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059774 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11e0aac9-5ea3-4ebc-bd38-37761997e5be" containerName="revoke-keys" Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059789 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="11e0aac9-5ea3-4ebc-bd38-37761997e5be" containerName="revoke-keys" Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059801 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" containerName="maas-api" Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059806 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" containerName="maas-api" Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059870 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="16ad6a8e-3ee6-4dba-8e2e-31d4ef51043e" containerName="maas-api" Jul 22 02:53:04.059900 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.059879 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="11e0aac9-5ea3-4ebc-bd38-37761997e5be" containerName="revoke-keys" Jul 22 02:53:04.063023 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.063004 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.065993 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.065971 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-6a145fe4-serving-cert\"" Jul 22 02:53:04.066122 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.065980 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:53:04.076648 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.076625 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:53:04.118302 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.118249 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.118503 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.118444 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.118771 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.118746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7zmj\" (UniqueName: \"kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.219431 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.219394 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.219611 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.219446 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.219611 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.219472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f7zmj\" (UniqueName: \"kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.219611 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:53:04.219592 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-serving-cert: secret "maas-api-e2e-mt-6a145fe4-serving-cert" not found Jul 22 02:53:04.219736 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:53:04.219655 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls podName:6beb882e-1cef-4d78-8397-2b83421a93e3 nodeName:}" failed. No retries permitted until 2026-07-22 02:53:04.719639022 +0000 UTC m=+2026.497714940 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls") pod "maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" (UID: "6beb882e-1cef-4d78-8397-2b83421a93e3") : secret "maas-api-e2e-mt-6a145fe4-serving-cert" not found Jul 22 02:53:04.219999 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.219980 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.228359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.228336 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7zmj\" (UniqueName: \"kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.724662 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.724583 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.727553 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.727523 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") pod \"maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:04.974473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:04.974437 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:05.122729 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:05.122695 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:53:05.125738 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:53:05.125709 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6beb882e_1cef_4d78_8397_2b83421a93e3.slice/crio-8a8e23bcc6bcf9ec6fb9fd3dbc998d0293aaa4f20ce5adf262c360fd824245ea WatchSource:0}: Error finding container 8a8e23bcc6bcf9ec6fb9fd3dbc998d0293aaa4f20ce5adf262c360fd824245ea: Status 404 returned error can't find the container with id 8a8e23bcc6bcf9ec6fb9fd3dbc998d0293aaa4f20ce5adf262c360fd824245ea Jul 22 02:53:05.996007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:05.995970 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" event={"ID":"6beb882e-1cef-4d78-8397-2b83421a93e3","Type":"ContainerStarted","Data":"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c"} Jul 22 02:53:05.996249 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:05.996019 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" event={"ID":"6beb882e-1cef-4d78-8397-2b83421a93e3","Type":"ContainerStarted","Data":"8a8e23bcc6bcf9ec6fb9fd3dbc998d0293aaa4f20ce5adf262c360fd824245ea"} Jul 22 02:53:05.996249 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:05.996048 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:12.007274 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:12.007245 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:53:12.027945 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:12.027891 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" podStartSLOduration=7.707429916 podStartE2EDuration="8.027876085s" totalImagesPullingTime="320.446169ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:53:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:53:06.034332584 +0000 UTC m=+2027.812408526" watchObservedRunningTime="2026-07-22 02:53:12.027876085 +0000 UTC m=+2033.805952024" Jul 22 02:53:46.784185 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.784091 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq"] Jul 22 02:53:46.787600 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.787581 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:46.790652 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.790635 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:53:46.798159 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.798115 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq"] Jul 22 02:53:46.871074 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.871031 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq\" (UID: \"b998fbc3-360a-44c0-b97f-a972d25a5b33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:46.971624 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.971584 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq\" (UID: \"b998fbc3-360a-44c0-b97f-a972d25a5b33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:46.972260 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:46.972240 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq\" (UID: \"b998fbc3-360a-44c0-b97f-a972d25a5b33\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:47.099544 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:47.099464 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:47.241092 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:47.241058 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq"] Jul 22 02:53:47.242308 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:53:47.242279 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb998fbc3_360a_44c0_b97f_a972d25a5b33.slice/crio-a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76 WatchSource:0}: Error finding container a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76: Status 404 returned error can't find the container with id a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76 Jul 22 02:53:48.234304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:48.234272 2570 generic.go:356] "Generic (PLEG): container finished" podID="b998fbc3-360a-44c0-b97f-a972d25a5b33" containerID="61b830e24c6f54a5d3f093987b4181252721e5c45d4840f2d0ed81f26dd923f7" exitCode=0 Jul 22 02:53:48.234751 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:48.234359 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" event={"ID":"b998fbc3-360a-44c0-b97f-a972d25a5b33","Type":"ContainerDied","Data":"61b830e24c6f54a5d3f093987b4181252721e5c45d4840f2d0ed81f26dd923f7"} Jul 22 02:53:48.234751 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:48.234416 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" event={"ID":"b998fbc3-360a-44c0-b97f-a972d25a5b33","Type":"ContainerStarted","Data":"a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76"} Jul 22 02:53:49.378304 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:49.378279 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:49.390602 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:49.390574 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca\") pod \"b998fbc3-360a-44c0-b97f-a972d25a5b33\" (UID: \"b998fbc3-360a-44c0-b97f-a972d25a5b33\") " Jul 22 02:53:49.390893 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:49.390873 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b998fbc3-360a-44c0-b97f-a972d25a5b33" (UID: "b998fbc3-360a-44c0-b97f-a972d25a5b33"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:53:49.491882 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:49.491804 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b998fbc3-360a-44c0-b97f-a972d25a5b33-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:53:50.246654 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:50.246629 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" Jul 22 02:53:50.246807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:50.246676 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq" event={"ID":"b998fbc3-360a-44c0-b97f-a972d25a5b33","Type":"ContainerDied","Data":"a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76"} Jul 22 02:53:50.246807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:53:50.246720 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a36d3e55386f22c578e8810c804999f452b1bb9b576bcc9246f9b54e347e5c76" Jul 22 02:54:02.348316 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.348278 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:54:02.348772 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.348504 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" podUID="6beb882e-1cef-4d78-8397-2b83421a93e3" containerName="maas-api" containerID="cri-o://60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c" gracePeriod=30 Jul 22 02:54:02.600946 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.600883 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:54:02.701628 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.701590 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7zmj\" (UniqueName: \"kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj\") pod \"6beb882e-1cef-4d78-8397-2b83421a93e3\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " Jul 22 02:54:02.701800 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.701645 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") pod \"6beb882e-1cef-4d78-8397-2b83421a93e3\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " Jul 22 02:54:02.701800 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.701694 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca\") pod \"6beb882e-1cef-4d78-8397-2b83421a93e3\" (UID: \"6beb882e-1cef-4d78-8397-2b83421a93e3\") " Jul 22 02:54:02.702124 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.702102 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6beb882e-1cef-4d78-8397-2b83421a93e3" (UID: "6beb882e-1cef-4d78-8397-2b83421a93e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:54:02.704001 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.703968 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj" (OuterVolumeSpecName: "kube-api-access-f7zmj") pod "6beb882e-1cef-4d78-8397-2b83421a93e3" (UID: "6beb882e-1cef-4d78-8397-2b83421a93e3"). InnerVolumeSpecName "kube-api-access-f7zmj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:54:02.704001 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.703966 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6beb882e-1cef-4d78-8397-2b83421a93e3" (UID: "6beb882e-1cef-4d78-8397-2b83421a93e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:54:02.803160 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.803104 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7zmj\" (UniqueName: \"kubernetes.io/projected/6beb882e-1cef-4d78-8397-2b83421a93e3-kube-api-access-f7zmj\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:54:02.803315 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.803180 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6beb882e-1cef-4d78-8397-2b83421a93e3-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:54:02.803315 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:02.803197 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6beb882e-1cef-4d78-8397-2b83421a93e3-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:54:03.316595 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.316563 2570 generic.go:356] "Generic (PLEG): container finished" podID="6beb882e-1cef-4d78-8397-2b83421a93e3" containerID="60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c" exitCode=0 Jul 22 02:54:03.316595 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.316602 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" event={"ID":"6beb882e-1cef-4d78-8397-2b83421a93e3","Type":"ContainerDied","Data":"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c"} Jul 22 02:54:03.316846 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.316609 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" Jul 22 02:54:03.316846 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.316641 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b" event={"ID":"6beb882e-1cef-4d78-8397-2b83421a93e3","Type":"ContainerDied","Data":"8a8e23bcc6bcf9ec6fb9fd3dbc998d0293aaa4f20ce5adf262c360fd824245ea"} Jul 22 02:54:03.316846 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.316661 2570 scope.go:117] "RemoveContainer" containerID="60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c" Jul 22 02:54:03.325775 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.325758 2570 scope.go:117] "RemoveContainer" containerID="60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c" Jul 22 02:54:03.326048 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:54:03.326030 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c\": container with ID starting with 60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c not found: ID does not exist" containerID="60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c" Jul 22 02:54:03.326087 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.326056 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c"} err="failed to get container status \"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c\": rpc error: code = NotFound desc = could not find container \"60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c\": container with ID starting with 60f18b9e469dee5217d671c0c0499171dbf2173c7eea88d81497aee7fa08267c not found: ID does not exist" Jul 22 02:54:03.336879 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.336853 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:54:03.340789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:03.340767 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-6a145fe4-85959dcc4b-kv96b"] Jul 22 02:54:04.748950 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:04.748915 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6beb882e-1cef-4d78-8397-2b83421a93e3" path="/var/lib/kubelet/pods/6beb882e-1cef-4d78-8397-2b83421a93e3/volumes" Jul 22 02:54:06.876794 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:06.876760 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq"] Jul 22 02:54:06.879413 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:06.879387 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-6a145fe4-zrpmq"] Jul 22 02:54:08.748517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:08.748485 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b998fbc3-360a-44c0-b97f-a972d25a5b33" path="/var/lib/kubelet/pods/b998fbc3-360a-44c0-b97f-a972d25a5b33/volumes" Jul 22 02:54:18.883941 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:18.883911 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:54:18.888345 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:18.888320 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:54:18.892878 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:18.892856 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:54:18.897025 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:18.897005 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:54:18.962612 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:54:18.962583 2570 scope.go:117] "RemoveContainer" containerID="91426b7f80e1ad4e539dc402c70dc86140b132f77c15ef082dd379894d760b8f" Jul 22 02:55:18.988158 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:55:18.988106 2570 scope.go:117] "RemoveContainer" containerID="59e364c5bc5e219a645d28100f31aeae9810376d706a61bceb21641c779d8b77" Jul 22 02:56:03.944215 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944182 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944517 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b998fbc3-360a-44c0-b97f-a972d25a5b33" containerName="revoke-keys" Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944528 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="b998fbc3-360a-44c0-b97f-a972d25a5b33" containerName="revoke-keys" Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944548 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6beb882e-1cef-4d78-8397-2b83421a93e3" containerName="maas-api" Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944554 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6beb882e-1cef-4d78-8397-2b83421a93e3" containerName="maas-api" Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944611 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="b998fbc3-360a-44c0-b97f-a972d25a5b33" containerName="revoke-keys" Jul 22 02:56:03.944670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.944618 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6beb882e-1cef-4d78-8397-2b83421a93e3" containerName="maas-api" Jul 22 02:56:03.947494 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.947470 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:03.950670 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.950646 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-fbb5cc-serving-cert\"" Jul 22 02:56:03.950785 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.950725 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:56:03.963176 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:03.963152 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:56:04.047884 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.047853 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.048081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.047890 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvgrt\" (UniqueName: \"kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.048081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.047971 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.148411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.148380 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.148411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.148414 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvgrt\" (UniqueName: \"kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.148697 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.148439 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.149119 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.149097 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.151019 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.150999 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.157570 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.157544 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvgrt\" (UniqueName: \"kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt\") pod \"maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.258347 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.258305 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:04.407964 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.407936 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:56:04.410013 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:56:04.409977 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b0a86df_6ed8_4b9a_97c7_c13a5364e352.slice/crio-b4fc2cded290db915341f6fd39ff78eb1c79f60ee41827cc3a7d7ca8ddb49614 WatchSource:0}: Error finding container b4fc2cded290db915341f6fd39ff78eb1c79f60ee41827cc3a7d7ca8ddb49614: Status 404 returned error can't find the container with id b4fc2cded290db915341f6fd39ff78eb1c79f60ee41827cc3a7d7ca8ddb49614 Jul 22 02:56:04.978010 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.977967 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" event={"ID":"4b0a86df-6ed8-4b9a-97c7-c13a5364e352","Type":"ContainerStarted","Data":"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5"} Jul 22 02:56:04.978439 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.978023 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" event={"ID":"4b0a86df-6ed8-4b9a-97c7-c13a5364e352","Type":"ContainerStarted","Data":"b4fc2cded290db915341f6fd39ff78eb1c79f60ee41827cc3a7d7ca8ddb49614"} Jul 22 02:56:04.978439 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:04.978120 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:05.003395 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:05.003320 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" podStartSLOduration=1.61367639 podStartE2EDuration="2.003298916s" totalImagesPullingTime="389.622526ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:04.999670453 +0000 UTC m=+2206.777746394" watchObservedRunningTime="2026-07-22 02:56:05.003298916 +0000 UTC m=+2206.781374858" Jul 22 02:56:10.989683 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:10.989655 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:56:11.807090 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.807055 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:56:11.811370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.811351 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:11.816628 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.816604 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-6873d4-serving-cert\"" Jul 22 02:56:11.827755 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.827718 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:56:11.905844 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.905807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:11.905844 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.905850 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:11.906078 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:11.905885 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc6nf\" (UniqueName: \"kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.006740 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.006708 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.007218 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.006770 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc6nf\" (UniqueName: \"kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.007218 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.006879 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.007218 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:56:12.006993 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-serving-cert: secret "maas-api-e2e-inf-b-6873d4-serving-cert" not found Jul 22 02:56:12.007218 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:56:12.007063 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls podName:6ae89e93-b162-4910-92af-f8686bfdf166 nodeName:}" failed. No retries permitted until 2026-07-22 02:56:12.507041796 +0000 UTC m=+2214.285117717 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls") pod "maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" (UID: "6ae89e93-b162-4910-92af-f8686bfdf166") : secret "maas-api-e2e-inf-b-6873d4-serving-cert" not found Jul 22 02:56:12.007412 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.007312 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.027795 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.027769 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc6nf\" (UniqueName: \"kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.510624 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.510590 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.513317 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.513284 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") pod \"maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.722060 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.722024 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:12.881838 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:12.881802 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:56:12.885324 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:56:12.885298 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ae89e93_b162_4910_92af_f8686bfdf166.slice/crio-b4db5b2dec832b5da4778ed3502d980fcd043f7d37df275d247f9bc4ceb4816e WatchSource:0}: Error finding container b4db5b2dec832b5da4778ed3502d980fcd043f7d37df275d247f9bc4ceb4816e: Status 404 returned error can't find the container with id b4db5b2dec832b5da4778ed3502d980fcd043f7d37df275d247f9bc4ceb4816e Jul 22 02:56:13.026036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:13.026001 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" event={"ID":"6ae89e93-b162-4910-92af-f8686bfdf166","Type":"ContainerStarted","Data":"b4db5b2dec832b5da4778ed3502d980fcd043f7d37df275d247f9bc4ceb4816e"} Jul 22 02:56:14.033509 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:14.033459 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" event={"ID":"6ae89e93-b162-4910-92af-f8686bfdf166","Type":"ContainerStarted","Data":"c63920a8d4e1f293a8d2f4cc1729aa7e681cbf8576999ef2fae443cea026765c"} Jul 22 02:56:14.033927 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:14.033668 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:56:14.054695 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:14.054637 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" podStartSLOduration=2.1243147000000002 podStartE2EDuration="3.054615678s" totalImagesPullingTime="930.300978ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:56:14.051935872 +0000 UTC m=+2215.830011825" watchObservedRunningTime="2026-07-22 02:56:14.054615678 +0000 UTC m=+2215.832691619" Jul 22 02:56:19.009492 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:19.009460 2570 scope.go:117] "RemoveContainer" containerID="8de1336b8067163efeee82a45783dc5c32b43999d4b4428a10ecfc74e96848dd" Jul 22 02:56:19.018388 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:19.018367 2570 scope.go:117] "RemoveContainer" containerID="cda7ec2a39bed6ee3740d9af7d1dd79d3fde9b744b348206d01b5ab9885c9f8f" Jul 22 02:56:20.045485 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:56:20.045457 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:57:52.280529 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.280487 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm"] Jul 22 02:57:52.284517 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.284495 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:52.288024 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.288003 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:57:52.300690 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.300663 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm"] Jul 22 02:57:52.409034 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.408998 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm\" (UID: \"3667f5ba-da45-4374-8ef4-a20121e70990\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:52.509821 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.509774 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm\" (UID: \"3667f5ba-da45-4374-8ef4-a20121e70990\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:52.510462 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.510441 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm\" (UID: \"3667f5ba-da45-4374-8ef4-a20121e70990\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:52.596396 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.596307 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:52.741156 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:52.741105 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm"] Jul 22 02:57:52.743471 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:57:52.743441 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3667f5ba_da45_4374_8ef4_a20121e70990.slice/crio-caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599 WatchSource:0}: Error finding container caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599: Status 404 returned error can't find the container with id caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599 Jul 22 02:57:53.577282 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:53.577251 2570 generic.go:356] "Generic (PLEG): container finished" podID="3667f5ba-da45-4374-8ef4-a20121e70990" containerID="cd63a3af64328e7d37e6a2c9fd06d1d3328bd8d9a5b58e2d496a535ff6e92cb2" exitCode=0 Jul 22 02:57:53.577694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:53.577336 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" event={"ID":"3667f5ba-da45-4374-8ef4-a20121e70990","Type":"ContainerDied","Data":"cd63a3af64328e7d37e6a2c9fd06d1d3328bd8d9a5b58e2d496a535ff6e92cb2"} Jul 22 02:57:53.577694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:53.577393 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" event={"ID":"3667f5ba-da45-4374-8ef4-a20121e70990","Type":"ContainerStarted","Data":"caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599"} Jul 22 02:57:54.721067 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:54.721038 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:54.831030 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:54.830994 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca\") pod \"3667f5ba-da45-4374-8ef4-a20121e70990\" (UID: \"3667f5ba-da45-4374-8ef4-a20121e70990\") " Jul 22 02:57:54.831395 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:54.831371 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3667f5ba-da45-4374-8ef4-a20121e70990" (UID: "3667f5ba-da45-4374-8ef4-a20121e70990"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:57:54.932473 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:54.932374 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3667f5ba-da45-4374-8ef4-a20121e70990-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:57:55.589539 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:55.589499 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" event={"ID":"3667f5ba-da45-4374-8ef4-a20121e70990","Type":"ContainerDied","Data":"caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599"} Jul 22 02:57:55.589539 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:55.589526 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm" Jul 22 02:57:55.589757 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:57:55.589585 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="caa3cfb1776c944972b2d4e747f9130ba8db9f623be33af6385eee97bd238599" Jul 22 02:58:02.684291 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:02.684250 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:58:02.684820 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:02.684578 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" podUID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" containerName="maas-api" containerID="cri-o://ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5" gracePeriod=30 Jul 22 02:58:03.143544 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.143520 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:58:03.204636 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.204598 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvgrt\" (UniqueName: \"kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt\") pod \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " Jul 22 02:58:03.204636 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.204638 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca\") pod \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " Jul 22 02:58:03.204916 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.204739 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls\") pod \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\" (UID: \"4b0a86df-6ed8-4b9a-97c7-c13a5364e352\") " Jul 22 02:58:03.205115 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.205088 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4b0a86df-6ed8-4b9a-97c7-c13a5364e352" (UID: "4b0a86df-6ed8-4b9a-97c7-c13a5364e352"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:03.207050 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.207014 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4b0a86df-6ed8-4b9a-97c7-c13a5364e352" (UID: "4b0a86df-6ed8-4b9a-97c7-c13a5364e352"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:58:03.207207 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.207019 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt" (OuterVolumeSpecName: "kube-api-access-qvgrt") pod "4b0a86df-6ed8-4b9a-97c7-c13a5364e352" (UID: "4b0a86df-6ed8-4b9a-97c7-c13a5364e352"). InnerVolumeSpecName "kube-api-access-qvgrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:03.306411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.306312 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:03.306411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.306347 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvgrt\" (UniqueName: \"kubernetes.io/projected/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-kube-api-access-qvgrt\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:03.306411 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.306356 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4b0a86df-6ed8-4b9a-97c7-c13a5364e352-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:03.633789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.633694 2570 generic.go:356] "Generic (PLEG): container finished" podID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" containerID="ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5" exitCode=0 Jul 22 02:58:03.633789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.633741 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" Jul 22 02:58:03.633789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.633777 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" event={"ID":"4b0a86df-6ed8-4b9a-97c7-c13a5364e352","Type":"ContainerDied","Data":"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5"} Jul 22 02:58:03.634036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.633826 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck" event={"ID":"4b0a86df-6ed8-4b9a-97c7-c13a5364e352","Type":"ContainerDied","Data":"b4fc2cded290db915341f6fd39ff78eb1c79f60ee41827cc3a7d7ca8ddb49614"} Jul 22 02:58:03.634036 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.633849 2570 scope.go:117] "RemoveContainer" containerID="ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5" Jul 22 02:58:03.646046 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.646025 2570 scope.go:117] "RemoveContainer" containerID="ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5" Jul 22 02:58:03.646384 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:58:03.646366 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5\": container with ID starting with ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5 not found: ID does not exist" containerID="ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5" Jul 22 02:58:03.646447 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.646393 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5"} err="failed to get container status \"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5\": rpc error: code = NotFound desc = could not find container \"ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5\": container with ID starting with ca680be3122da339c4e752fb24d5eb2e730db3019ee2c16c4f5e07b296f617e5 not found: ID does not exist" Jul 22 02:58:03.658904 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.658879 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:58:03.662382 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:03.662360 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-fbb5cc-7447486575-tf4ck"] Jul 22 02:58:04.749359 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:04.749323 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" path="/var/lib/kubelet/pods/4b0a86df-6ed8-4b9a-97c7-c13a5364e352/volumes" Jul 22 02:58:07.484783 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:07.484740 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm"] Jul 22 02:58:07.492010 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:07.491982 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-fbb5cc-7jfzm"] Jul 22 02:58:08.749079 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:08.749036 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3667f5ba-da45-4374-8ef4-a20121e70990" path="/var/lib/kubelet/pods/3667f5ba-da45-4374-8ef4-a20121e70990/volumes" Jul 22 02:58:15.185084 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185047 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm"] Jul 22 02:58:15.185496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185424 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3667f5ba-da45-4374-8ef4-a20121e70990" containerName="revoke-keys" Jul 22 02:58:15.185496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185436 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="3667f5ba-da45-4374-8ef4-a20121e70990" containerName="revoke-keys" Jul 22 02:58:15.185496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185458 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" containerName="maas-api" Jul 22 02:58:15.185496 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185463 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" containerName="maas-api" Jul 22 02:58:15.185629 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185524 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b0a86df-6ed8-4b9a-97c7-c13a5364e352" containerName="maas-api" Jul 22 02:58:15.185629 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.185530 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="3667f5ba-da45-4374-8ef4-a20121e70990" containerName="revoke-keys" Jul 22 02:58:15.189861 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.189841 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:15.193602 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.193582 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 02:58:15.201810 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.201788 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm"] Jul 22 02:58:15.311007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.310980 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm\" (UID: \"c7304066-c1a0-465a-8b2a-5f6dff349bce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:15.411626 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.411573 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm\" (UID: \"c7304066-c1a0-465a-8b2a-5f6dff349bce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:15.412262 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.412240 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm\" (UID: \"c7304066-c1a0-465a-8b2a-5f6dff349bce\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:15.501052 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.501018 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:15.650979 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.650949 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm"] Jul 22 02:58:15.657199 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:58:15.657169 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7304066_c1a0_465a_8b2a_5f6dff349bce.slice/crio-2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876 WatchSource:0}: Error finding container 2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876: Status 404 returned error can't find the container with id 2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876 Jul 22 02:58:15.699365 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:15.699339 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" event={"ID":"c7304066-c1a0-465a-8b2a-5f6dff349bce","Type":"ContainerStarted","Data":"2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876"} Jul 22 02:58:16.706807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:16.706774 2570 generic.go:356] "Generic (PLEG): container finished" podID="c7304066-c1a0-465a-8b2a-5f6dff349bce" containerID="d6439ced39a43167da1c14a157c57d9cea110823ec7b1a351ab8a1ad46674238" exitCode=0 Jul 22 02:58:16.707239 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:16.706868 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" event={"ID":"c7304066-c1a0-465a-8b2a-5f6dff349bce","Type":"ContainerDied","Data":"d6439ced39a43167da1c14a157c57d9cea110823ec7b1a351ab8a1ad46674238"} Jul 22 02:58:17.853463 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:17.853433 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:17.934410 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:17.934372 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca\") pod \"c7304066-c1a0-465a-8b2a-5f6dff349bce\" (UID: \"c7304066-c1a0-465a-8b2a-5f6dff349bce\") " Jul 22 02:58:17.934764 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:17.934744 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "c7304066-c1a0-465a-8b2a-5f6dff349bce" (UID: "c7304066-c1a0-465a-8b2a-5f6dff349bce"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:18.035797 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:18.035711 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/c7304066-c1a0-465a-8b2a-5f6dff349bce-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:18.719668 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:18.719633 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" event={"ID":"c7304066-c1a0-465a-8b2a-5f6dff349bce","Type":"ContainerDied","Data":"2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876"} Jul 22 02:58:18.719848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:18.719685 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2c24ca3a1ff3e94adc4067b89c7e1df42a9b9d75a1d1d16e6cd2b868685c7876" Jul 22 02:58:18.719848 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:18.719645 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm" Jul 22 02:58:19.053246 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:19.053168 2570 scope.go:117] "RemoveContainer" containerID="83cfb146c82db2fe8b0c9419b16c4b16468748da5031470ff560160ea0c82a86" Jul 22 02:58:25.593956 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.593922 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:58:25.594553 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.594198 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" podUID="6ae89e93-b162-4910-92af-f8686bfdf166" containerName="maas-api" containerID="cri-o://c63920a8d4e1f293a8d2f4cc1729aa7e681cbf8576999ef2fae443cea026765c" gracePeriod=30 Jul 22 02:58:25.759660 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.759619 2570 generic.go:356] "Generic (PLEG): container finished" podID="6ae89e93-b162-4910-92af-f8686bfdf166" containerID="c63920a8d4e1f293a8d2f4cc1729aa7e681cbf8576999ef2fae443cea026765c" exitCode=0 Jul 22 02:58:25.759825 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.759686 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" event={"ID":"6ae89e93-b162-4910-92af-f8686bfdf166","Type":"ContainerDied","Data":"c63920a8d4e1f293a8d2f4cc1729aa7e681cbf8576999ef2fae443cea026765c"} Jul 22 02:58:25.853919 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.853856 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:58:25.915195 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.915159 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca\") pod \"6ae89e93-b162-4910-92af-f8686bfdf166\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " Jul 22 02:58:25.915195 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.915199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc6nf\" (UniqueName: \"kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf\") pod \"6ae89e93-b162-4910-92af-f8686bfdf166\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " Jul 22 02:58:25.915419 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.915241 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") pod \"6ae89e93-b162-4910-92af-f8686bfdf166\" (UID: \"6ae89e93-b162-4910-92af-f8686bfdf166\") " Jul 22 02:58:25.915541 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.915514 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6ae89e93-b162-4910-92af-f8686bfdf166" (UID: "6ae89e93-b162-4910-92af-f8686bfdf166"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:58:25.917511 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.917482 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6ae89e93-b162-4910-92af-f8686bfdf166" (UID: "6ae89e93-b162-4910-92af-f8686bfdf166"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:58:25.917620 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:25.917516 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf" (OuterVolumeSpecName: "kube-api-access-bc6nf") pod "6ae89e93-b162-4910-92af-f8686bfdf166" (UID: "6ae89e93-b162-4910-92af-f8686bfdf166"). InnerVolumeSpecName "kube-api-access-bc6nf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:58:26.016162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.016096 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6ae89e93-b162-4910-92af-f8686bfdf166-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:26.016162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.016129 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6ae89e93-b162-4910-92af-f8686bfdf166-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:26.016162 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.016165 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc6nf\" (UniqueName: \"kubernetes.io/projected/6ae89e93-b162-4910-92af-f8686bfdf166-kube-api-access-bc6nf\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:58:26.766081 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.766046 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" Jul 22 02:58:26.766506 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.766056 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7" event={"ID":"6ae89e93-b162-4910-92af-f8686bfdf166","Type":"ContainerDied","Data":"b4db5b2dec832b5da4778ed3502d980fcd043f7d37df275d247f9bc4ceb4816e"} Jul 22 02:58:26.766506 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.766190 2570 scope.go:117] "RemoveContainer" containerID="c63920a8d4e1f293a8d2f4cc1729aa7e681cbf8576999ef2fae443cea026765c" Jul 22 02:58:26.785189 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.785161 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:58:26.787289 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:26.787270 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-6873d4-7746c4647f-n22b7"] Jul 22 02:58:28.748229 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:28.748193 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ae89e93-b162-4910-92af-f8686bfdf166" path="/var/lib/kubelet/pods/6ae89e93-b162-4910-92af-f8686bfdf166/volumes" Jul 22 02:58:30.295678 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:30.295641 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm"] Jul 22 02:58:30.298985 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:30.298959 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-6873d4-97wtm"] Jul 22 02:58:30.748600 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:30.748567 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7304066-c1a0-465a-8b2a-5f6dff349bce" path="/var/lib/kubelet/pods/c7304066-c1a0-465a-8b2a-5f6dff349bce/volumes" Jul 22 02:58:41.717877 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.717837 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:58:41.718329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718276 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ae89e93-b162-4910-92af-f8686bfdf166" containerName="maas-api" Jul 22 02:58:41.718329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718292 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ae89e93-b162-4910-92af-f8686bfdf166" containerName="maas-api" Jul 22 02:58:41.718329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718301 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7304066-c1a0-465a-8b2a-5f6dff349bce" containerName="revoke-keys" Jul 22 02:58:41.718329 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718308 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7304066-c1a0-465a-8b2a-5f6dff349bce" containerName="revoke-keys" Jul 22 02:58:41.718460 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718372 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ae89e93-b162-4910-92af-f8686bfdf166" containerName="maas-api" Jul 22 02:58:41.718460 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.718380 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7304066-c1a0-465a-8b2a-5f6dff349bce" containerName="revoke-keys" Jul 22 02:58:41.722763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.722745 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.725658 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.725640 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-3f8fef-serving-cert\"" Jul 22 02:58:41.726342 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.726328 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 02:58:41.767267 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.767240 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:58:41.849818 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.849783 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.850007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.849833 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.850007 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.849943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmv6x\" (UniqueName: \"kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.950807 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.950765 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tmv6x\" (UniqueName: \"kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.950996 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.950847 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.951073 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.951016 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.951199 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:58:41.951179 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-serving-cert: secret "maas-api-e2e-api-a-3f8fef-serving-cert" not found Jul 22 02:58:41.951275 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:58:41.951249 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls podName:8f23741b-ef6b-4c32-a10e-16897041a131 nodeName:}" failed. No retries permitted until 2026-07-22 02:58:42.451227839 +0000 UTC m=+2364.229303759 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls") pod "maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" (UID: "8f23741b-ef6b-4c32-a10e-16897041a131") : secret "maas-api-e2e-api-a-3f8fef-serving-cert" not found Jul 22 02:58:41.951577 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.951556 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:41.972057 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:41.971991 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmv6x\" (UniqueName: \"kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:42.456825 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.456788 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:42.459424 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.459402 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") pod \"maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:42.633212 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.633178 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:42.784631 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.784602 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:58:42.786361 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:58:42.786337 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f23741b_ef6b_4c32_a10e_16897041a131.slice/crio-e5391653db4f647ce7a76fab85550c374ca1230a44a8e285c7be635746ebeeca WatchSource:0}: Error finding container e5391653db4f647ce7a76fab85550c374ca1230a44a8e285c7be635746ebeeca: Status 404 returned error can't find the container with id e5391653db4f647ce7a76fab85550c374ca1230a44a8e285c7be635746ebeeca Jul 22 02:58:42.787737 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.787717 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 22 02:58:42.860988 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:42.860953 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" event={"ID":"8f23741b-ef6b-4c32-a10e-16897041a131","Type":"ContainerStarted","Data":"e5391653db4f647ce7a76fab85550c374ca1230a44a8e285c7be635746ebeeca"} Jul 22 02:58:43.868603 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:43.868565 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" event={"ID":"8f23741b-ef6b-4c32-a10e-16897041a131","Type":"ContainerStarted","Data":"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4"} Jul 22 02:58:43.869069 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:43.868620 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:43.892059 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:43.891998 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" podStartSLOduration=2.444648715 podStartE2EDuration="2.891982918s" totalImagesPullingTime="447.334203ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:43.890723671 +0000 UTC m=+2365.668799623" watchObservedRunningTime="2026-07-22 02:58:43.891982918 +0000 UTC m=+2365.670058907" Jul 22 02:58:49.792624 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.792590 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:58:49.796354 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.796336 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.801398 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.801379 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-dac71a-serving-cert\"" Jul 22 02:58:49.813992 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.813965 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:58:49.822791 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.822771 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.822897 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.822798 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.822897 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.822817 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7fhc\" (UniqueName: \"kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.878951 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.878927 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:58:49.923763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.923678 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.923763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.923721 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.923763 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.923745 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7fhc\" (UniqueName: \"kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.924683 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.924643 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.926419 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.926401 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:49.947639 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:49.947607 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7fhc\" (UniqueName: \"kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc\") pod \"maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:50.106024 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.105917 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:50.260401 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.260375 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:58:50.262649 ip-10-0-142-149 kubenswrapper[2570]: W0722 02:58:50.262622 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5b3783a_5d15_4e9f_8119_0c845f2a2f12.slice/crio-062fb3017f9da543a68761f12bbc0c3b416850de8005894c217339028155f77f WatchSource:0}: Error finding container 062fb3017f9da543a68761f12bbc0c3b416850de8005894c217339028155f77f: Status 404 returned error can't find the container with id 062fb3017f9da543a68761f12bbc0c3b416850de8005894c217339028155f77f Jul 22 02:58:50.910743 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.910697 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" event={"ID":"c5b3783a-5d15-4e9f-8119-0c845f2a2f12","Type":"ContainerStarted","Data":"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9"} Jul 22 02:58:50.911222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.910852 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:58:50.911222 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.910871 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" event={"ID":"c5b3783a-5d15-4e9f-8119-0c845f2a2f12","Type":"ContainerStarted","Data":"062fb3017f9da543a68761f12bbc0c3b416850de8005894c217339028155f77f"} Jul 22 02:58:50.941272 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:50.941218 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" podStartSLOduration=1.409103406 podStartE2EDuration="1.941201896s" totalImagesPullingTime="532.09849ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 02:58:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 02:58:50.939922568 +0000 UTC m=+2372.717998518" watchObservedRunningTime="2026-07-22 02:58:50.941201896 +0000 UTC m=+2372.719277838" Jul 22 02:58:56.921122 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:58:56.921088 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:59:18.071899 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.071848 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:59:18.072533 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.072379 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" podUID="8f23741b-ef6b-4c32-a10e-16897041a131" containerName="maas-api" containerID="cri-o://6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4" gracePeriod=30 Jul 22 02:59:18.328536 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.328466 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:59:18.481042 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.481002 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca\") pod \"8f23741b-ef6b-4c32-a10e-16897041a131\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " Jul 22 02:59:18.481250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.481113 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tmv6x\" (UniqueName: \"kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x\") pod \"8f23741b-ef6b-4c32-a10e-16897041a131\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " Jul 22 02:59:18.481250 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.481177 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") pod \"8f23741b-ef6b-4c32-a10e-16897041a131\" (UID: \"8f23741b-ef6b-4c32-a10e-16897041a131\") " Jul 22 02:59:18.481494 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.481462 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8f23741b-ef6b-4c32-a10e-16897041a131" (UID: "8f23741b-ef6b-4c32-a10e-16897041a131"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:59:18.483456 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.483433 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x" (OuterVolumeSpecName: "kube-api-access-tmv6x") pod "8f23741b-ef6b-4c32-a10e-16897041a131" (UID: "8f23741b-ef6b-4c32-a10e-16897041a131"). InnerVolumeSpecName "kube-api-access-tmv6x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:59:18.483456 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.483441 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8f23741b-ef6b-4c32-a10e-16897041a131" (UID: "8f23741b-ef6b-4c32-a10e-16897041a131"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:59:18.581909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.581820 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tmv6x\" (UniqueName: \"kubernetes.io/projected/8f23741b-ef6b-4c32-a10e-16897041a131-kube-api-access-tmv6x\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:18.581909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.581853 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8f23741b-ef6b-4c32-a10e-16897041a131-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:18.581909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.581862 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8f23741b-ef6b-4c32-a10e-16897041a131-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:18.920319 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.920242 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:59:18.925909 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.925888 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 02:59:18.928771 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.928749 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:59:18.934700 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:18.934679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 02:59:19.068205 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.068169 2570 generic.go:356] "Generic (PLEG): container finished" podID="8f23741b-ef6b-4c32-a10e-16897041a131" containerID="6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4" exitCode=0 Jul 22 02:59:19.068370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.068227 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" event={"ID":"8f23741b-ef6b-4c32-a10e-16897041a131","Type":"ContainerDied","Data":"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4"} Jul 22 02:59:19.068370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.068268 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" Jul 22 02:59:19.068370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.068281 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87" event={"ID":"8f23741b-ef6b-4c32-a10e-16897041a131","Type":"ContainerDied","Data":"e5391653db4f647ce7a76fab85550c374ca1230a44a8e285c7be635746ebeeca"} Jul 22 02:59:19.068370 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.068310 2570 scope.go:117] "RemoveContainer" containerID="6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4" Jul 22 02:59:19.080620 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.080423 2570 scope.go:117] "RemoveContainer" containerID="6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4" Jul 22 02:59:19.080872 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:59:19.080734 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4\": container with ID starting with 6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4 not found: ID does not exist" containerID="6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4" Jul 22 02:59:19.080872 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.080760 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4"} err="failed to get container status \"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4\": rpc error: code = NotFound desc = could not find container \"6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4\": container with ID starting with 6bad781f32d66cf8bfadc26ca32826284108de3cade535f82551cdb8201af3d4 not found: ID does not exist" Jul 22 02:59:19.087633 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.087605 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:59:19.092901 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:19.092881 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-3f8fef-56bfb9f565-4cz87"] Jul 22 02:59:20.749870 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:20.749832 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8f23741b-ef6b-4c32-a10e-16897041a131" path="/var/lib/kubelet/pods/8f23741b-ef6b-4c32-a10e-16897041a131/volumes" Jul 22 02:59:45.169429 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.169351 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:59:45.170051 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.169725 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" podUID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" containerName="maas-api" containerID="cri-o://fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9" gracePeriod=30 Jul 22 02:59:45.420176 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.420108 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:59:45.518789 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.518757 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca\") pod \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " Jul 22 02:59:45.518975 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.518848 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls\") pod \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " Jul 22 02:59:45.518975 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.518895 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7fhc\" (UniqueName: \"kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc\") pod \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\" (UID: \"c5b3783a-5d15-4e9f-8119-0c845f2a2f12\") " Jul 22 02:59:45.519234 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.519207 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "c5b3783a-5d15-4e9f-8119-0c845f2a2f12" (UID: "c5b3783a-5d15-4e9f-8119-0c845f2a2f12"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 02:59:45.521065 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.521043 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc" (OuterVolumeSpecName: "kube-api-access-d7fhc") pod "c5b3783a-5d15-4e9f-8119-0c845f2a2f12" (UID: "c5b3783a-5d15-4e9f-8119-0c845f2a2f12"). InnerVolumeSpecName "kube-api-access-d7fhc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 02:59:45.521178 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.521158 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "c5b3783a-5d15-4e9f-8119-0c845f2a2f12" (UID: "c5b3783a-5d15-4e9f-8119-0c845f2a2f12"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 02:59:45.619748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.619717 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7fhc\" (UniqueName: \"kubernetes.io/projected/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-kube-api-access-d7fhc\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:45.619748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.619746 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:45.619748 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:45.619757 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/c5b3783a-5d15-4e9f-8119-0c845f2a2f12-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 02:59:46.209292 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.209253 2570 generic.go:356] "Generic (PLEG): container finished" podID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" containerID="fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9" exitCode=0 Jul 22 02:59:46.209694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.209316 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" event={"ID":"c5b3783a-5d15-4e9f-8119-0c845f2a2f12","Type":"ContainerDied","Data":"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9"} Jul 22 02:59:46.209694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.209355 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" event={"ID":"c5b3783a-5d15-4e9f-8119-0c845f2a2f12","Type":"ContainerDied","Data":"062fb3017f9da543a68761f12bbc0c3b416850de8005894c217339028155f77f"} Jul 22 02:59:46.209694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.209375 2570 scope.go:117] "RemoveContainer" containerID="fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9" Jul 22 02:59:46.209694 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.209316 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj" Jul 22 02:59:46.219006 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.218988 2570 scope.go:117] "RemoveContainer" containerID="fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9" Jul 22 02:59:46.219301 ip-10-0-142-149 kubenswrapper[2570]: E0722 02:59:46.219280 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9\": container with ID starting with fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9 not found: ID does not exist" containerID="fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9" Jul 22 02:59:46.219381 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.219310 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9"} err="failed to get container status \"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9\": rpc error: code = NotFound desc = could not find container \"fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9\": container with ID starting with fd20ac7f2a653b61db6d703261f99e556ad7b78843f90ffde49a79846a4cb2f9 not found: ID does not exist" Jul 22 02:59:46.252294 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.252261 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:59:46.260200 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.260173 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-dac71a-d89cc77cc-7cccj"] Jul 22 02:59:46.748808 ip-10-0-142-149 kubenswrapper[2570]: I0722 02:59:46.748776 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" path="/var/lib/kubelet/pods/c5b3783a-5d15-4e9f-8119-0c845f2a2f12/volumes" Jul 22 03:00:00.153618 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153580 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz"] Jul 22 03:00:00.154002 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153908 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" containerName="maas-api" Jul 22 03:00:00.154002 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153918 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" containerName="maas-api" Jul 22 03:00:00.154002 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153934 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8f23741b-ef6b-4c32-a10e-16897041a131" containerName="maas-api" Jul 22 03:00:00.154002 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153939 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="8f23741b-ef6b-4c32-a10e-16897041a131" containerName="maas-api" Jul 22 03:00:00.154002 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.153999 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="c5b3783a-5d15-4e9f-8119-0c845f2a2f12" containerName="maas-api" Jul 22 03:00:00.154196 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.154009 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="8f23741b-ef6b-4c32-a10e-16897041a131" containerName="maas-api" Jul 22 03:00:00.157071 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.157055 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" Jul 22 03:00:00.160069 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.160051 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 03:00:00.166381 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.166362 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" Jul 22 03:00:00.172430 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.172400 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz"] Jul 22 03:00:00.318831 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:00.318781 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz"] Jul 22 03:00:00.321701 ip-10-0-142-149 kubenswrapper[2570]: W0722 03:00:00.321675 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod188ffe3e_0974_4274_bcff_b16c50b30f6f.slice/crio-1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103 WatchSource:0}: Error finding container 1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103: Status 404 returned error can't find the container with id 1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103 Jul 22 03:00:01.293186 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:01.293155 2570 generic.go:356] "Generic (PLEG): container finished" podID="188ffe3e-0974-4274-bcff-b16c50b30f6f" containerID="5243e226135423220b77dd98875c6f874f8204f2bb1fe7f6d2044718b4193655" exitCode=0 Jul 22 03:00:01.293586 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:01.293226 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" event={"ID":"188ffe3e-0974-4274-bcff-b16c50b30f6f","Type":"ContainerDied","Data":"5243e226135423220b77dd98875c6f874f8204f2bb1fe7f6d2044718b4193655"} Jul 22 03:00:01.293586 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:01.293281 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" event={"ID":"188ffe3e-0974-4274-bcff-b16c50b30f6f","Type":"ContainerStarted","Data":"1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103"} Jul 22 03:00:02.209738 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.209700 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:00:02.213234 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.213211 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.220568 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.220545 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-q9rm7\"" Jul 22 03:00:02.220689 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.220573 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-995f8f-serving-cert\"" Jul 22 03:00:02.260684 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.260649 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:00:02.271195 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.271152 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.271392 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.271229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4z88\" (UniqueName: \"kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.271392 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.271273 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.372742 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.372709 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.373294 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.372780 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.373294 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.372828 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4z88\" (UniqueName: \"kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.373733 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.373711 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.375428 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.375402 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.393724 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.393697 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4z88\" (UniqueName: \"kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88\") pod \"maas-api-e2e-shared-a-995f8f-598fb698f-crdxp\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.442848 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.442821 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" Jul 22 03:00:02.523767 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.523686 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:02.699492 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:02.696984 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:00:02.699928 ip-10-0-142-149 kubenswrapper[2570]: W0722 03:00:02.699904 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod800a1c35_a149_4d9e_aa01_3e36674855ca.slice/crio-eb8ae2663869060118810ef72819e0d4d777536138e9c67d0d18e6a3d8a8e972 WatchSource:0}: Error finding container eb8ae2663869060118810ef72819e0d4d777536138e9c67d0d18e6a3d8a8e972: Status 404 returned error can't find the container with id eb8ae2663869060118810ef72819e0d4d777536138e9c67d0d18e6a3d8a8e972 Jul 22 03:00:03.304555 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:03.304524 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" event={"ID":"188ffe3e-0974-4274-bcff-b16c50b30f6f","Type":"ContainerDied","Data":"1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103"} Jul 22 03:00:03.304680 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:03.304553 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29744820-wbqwz" Jul 22 03:00:03.304680 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:03.304584 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1cb0320c187d78309a4226c4b92dd195d15700088b4c77e6985d3310aca11103" Jul 22 03:00:03.305821 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:03.305799 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" event={"ID":"800a1c35-a149-4d9e-aa01-3e36674855ca","Type":"ContainerStarted","Data":"eb8ae2663869060118810ef72819e0d4d777536138e9c67d0d18e6a3d8a8e972"} Jul 22 03:00:04.313807 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:04.313774 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" event={"ID":"800a1c35-a149-4d9e-aa01-3e36674855ca","Type":"ContainerStarted","Data":"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666"} Jul 22 03:00:04.314342 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:04.313895 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:04.353064 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:04.353003 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" podStartSLOduration=1.7614011280000001 podStartE2EDuration="2.352985086s" totalImagesPullingTime="591.583958ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:00:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:04.351890722 +0000 UTC m=+2446.129966663" watchObservedRunningTime="2026-07-22 03:00:04.352985086 +0000 UTC m=+2446.131061026" Jul 22 03:00:10.324613 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.324580 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:00:10.325116 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.325091 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="188ffe3e-0974-4274-bcff-b16c50b30f6f" containerName="cleanup" Jul 22 03:00:10.325116 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.325114 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="188ffe3e-0974-4274-bcff-b16c50b30f6f" containerName="cleanup" Jul 22 03:00:10.325342 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.325276 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="188ffe3e-0974-4274-bcff-b16c50b30f6f" containerName="cleanup" Jul 22 03:00:10.328644 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.328626 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:00:10.328756 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.328686 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.341888 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.341866 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-bb8c9d-serving-cert\"" Jul 22 03:00:10.350305 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.350278 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:00:10.447795 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.447763 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.447969 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.447838 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.447969 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.447867 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8n5r\" (UniqueName: \"kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.549204 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.549164 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.549422 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.549224 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.549422 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.549255 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8n5r\" (UniqueName: \"kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.549422 ip-10-0-142-149 kubenswrapper[2570]: E0722 03:00:10.549340 2570 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-serving-cert: secret "maas-api-e2e-shared-b-bb8c9d-serving-cert" not found Jul 22 03:00:10.549422 ip-10-0-142-149 kubenswrapper[2570]: E0722 03:00:10.549413 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls podName:136bfa59-5551-41e5-a587-e0804e089283 nodeName:}" failed. No retries permitted until 2026-07-22 03:00:11.049392737 +0000 UTC m=+2452.827468656 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls") pod "maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" (UID: "136bfa59-5551-41e5-a587-e0804e089283") : secret "maas-api-e2e-shared-b-bb8c9d-serving-cert" not found Jul 22 03:00:10.549857 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.549836 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:10.575666 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:10.575600 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8n5r\" (UniqueName: \"kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:11.055349 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:11.055316 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:11.058019 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:11.057984 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") pod \"maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:11.239500 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:11.239462 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:11.425714 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:11.425691 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:00:11.427980 ip-10-0-142-149 kubenswrapper[2570]: W0722 03:00:11.427953 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod136bfa59_5551_41e5_a587_e0804e089283.slice/crio-9ffbbe06b06eadea9990d012dd5cdcf877bc6dcf7e8038cbb40b837584d95362 WatchSource:0}: Error finding container 9ffbbe06b06eadea9990d012dd5cdcf877bc6dcf7e8038cbb40b837584d95362: Status 404 returned error can't find the container with id 9ffbbe06b06eadea9990d012dd5cdcf877bc6dcf7e8038cbb40b837584d95362 Jul 22 03:00:12.361542 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:12.361495 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" event={"ID":"136bfa59-5551-41e5-a587-e0804e089283","Type":"ContainerStarted","Data":"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6"} Jul 22 03:00:12.361754 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:12.361583 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" event={"ID":"136bfa59-5551-41e5-a587-e0804e089283","Type":"ContainerStarted","Data":"9ffbbe06b06eadea9990d012dd5cdcf877bc6dcf7e8038cbb40b837584d95362"} Jul 22 03:00:12.361754 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:12.361607 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:12.387866 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:12.387807 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" podStartSLOduration=2.037358438 podStartE2EDuration="2.387791438s" totalImagesPullingTime="350.433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:00:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:00:12.38395151 +0000 UTC m=+2454.162027451" watchObservedRunningTime="2026-07-22 03:00:12.387791438 +0000 UTC m=+2454.165867380" Jul 22 03:00:18.373596 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:18.373569 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:00:19.104906 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:00:19.104869 2570 scope.go:117] "RemoveContainer" containerID="61b830e24c6f54a5d3f093987b4181252721e5c45d4840f2d0ed81f26dd923f7" Jul 22 03:04:18.955849 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:18.955762 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 03:04:18.964729 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:18.964704 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 03:04:18.965628 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:18.965606 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 03:04:18.979164 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:18.975318 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-7wdfd_0bce0e38-56ae-41d0-82c4-2f22e597c6d8/ovn-acl-logging/0.log" Jul 22 03:04:19.158783 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:19.158748 2570 scope.go:117] "RemoveContainer" containerID="d6439ced39a43167da1c14a157c57d9cea110823ec7b1a351ab8a1ad46674238" Jul 22 03:04:19.167783 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:04:19.167763 2570 scope.go:117] "RemoveContainer" containerID="cd63a3af64328e7d37e6a2c9fd06d1d3328bd8d9a5b58e2d496a535ff6e92cb2" Jul 22 03:06:40.623731 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.623699 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:06:40.624159 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.623976 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" podUID="800a1c35-a149-4d9e-aa01-3e36674855ca" containerName="maas-api" containerID="cri-o://aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666" gracePeriod=30 Jul 22 03:06:40.876387 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.876319 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:06:40.966120 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.966083 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls\") pod \"800a1c35-a149-4d9e-aa01-3e36674855ca\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " Jul 22 03:06:40.966359 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.966180 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4z88\" (UniqueName: \"kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88\") pod \"800a1c35-a149-4d9e-aa01-3e36674855ca\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " Jul 22 03:06:40.966359 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.966205 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca\") pod \"800a1c35-a149-4d9e-aa01-3e36674855ca\" (UID: \"800a1c35-a149-4d9e-aa01-3e36674855ca\") " Jul 22 03:06:40.966600 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.966572 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "800a1c35-a149-4d9e-aa01-3e36674855ca" (UID: "800a1c35-a149-4d9e-aa01-3e36674855ca"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:06:40.968385 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.968360 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "800a1c35-a149-4d9e-aa01-3e36674855ca" (UID: "800a1c35-a149-4d9e-aa01-3e36674855ca"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:06:40.968472 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:40.968400 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88" (OuterVolumeSpecName: "kube-api-access-n4z88") pod "800a1c35-a149-4d9e-aa01-3e36674855ca" (UID: "800a1c35-a149-4d9e-aa01-3e36674855ca"). InnerVolumeSpecName "kube-api-access-n4z88". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:06:41.066774 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.066740 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/800a1c35-a149-4d9e-aa01-3e36674855ca-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:06:41.066774 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.066770 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4z88\" (UniqueName: \"kubernetes.io/projected/800a1c35-a149-4d9e-aa01-3e36674855ca-kube-api-access-n4z88\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:06:41.066774 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.066780 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/800a1c35-a149-4d9e-aa01-3e36674855ca-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:06:41.454311 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.454277 2570 generic.go:356] "Generic (PLEG): container finished" podID="800a1c35-a149-4d9e-aa01-3e36674855ca" containerID="aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666" exitCode=0 Jul 22 03:06:41.454474 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.454316 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" Jul 22 03:06:41.454474 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.454349 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" event={"ID":"800a1c35-a149-4d9e-aa01-3e36674855ca","Type":"ContainerDied","Data":"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666"} Jul 22 03:06:41.454474 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.454398 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp" event={"ID":"800a1c35-a149-4d9e-aa01-3e36674855ca","Type":"ContainerDied","Data":"eb8ae2663869060118810ef72819e0d4d777536138e9c67d0d18e6a3d8a8e972"} Jul 22 03:06:41.454474 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.454421 2570 scope.go:117] "RemoveContainer" containerID="aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666" Jul 22 03:06:41.464323 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.464253 2570 scope.go:117] "RemoveContainer" containerID="aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666" Jul 22 03:06:41.464541 ip-10-0-142-149 kubenswrapper[2570]: E0722 03:06:41.464515 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666\": container with ID starting with aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666 not found: ID does not exist" containerID="aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666" Jul 22 03:06:41.464640 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.464549 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666"} err="failed to get container status \"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666\": rpc error: code = NotFound desc = could not find container \"aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666\": container with ID starting with aef83152e5d299c4dccde232733258de6445e4a1851283ad81950c890b0b4666 not found: ID does not exist" Jul 22 03:06:41.480433 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.480408 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:06:41.487607 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:41.487583 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-995f8f-598fb698f-crdxp"] Jul 22 03:06:42.748652 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:42.748617 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="800a1c35-a149-4d9e-aa01-3e36674855ca" path="/var/lib/kubelet/pods/800a1c35-a149-4d9e-aa01-3e36674855ca/volumes" Jul 22 03:06:52.576560 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.576527 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh"] Jul 22 03:06:52.577210 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.577050 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="800a1c35-a149-4d9e-aa01-3e36674855ca" containerName="maas-api" Jul 22 03:06:52.577210 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.577072 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="800a1c35-a149-4d9e-aa01-3e36674855ca" containerName="maas-api" Jul 22 03:06:52.577210 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.577196 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="800a1c35-a149-4d9e-aa01-3e36674855ca" containerName="maas-api" Jul 22 03:06:52.581503 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.581480 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:52.584332 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.584306 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-rd6jc\"" Jul 22 03:06:52.587912 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.587888 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh"] Jul 22 03:06:52.675536 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.675503 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh\" (UID: \"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:52.776886 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.776849 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh\" (UID: \"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:52.777517 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.777495 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh\" (UID: \"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:52.894334 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:52.894233 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:53.039839 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:53.038946 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh"] Jul 22 03:06:53.521230 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:53.521115 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" event={"ID":"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a","Type":"ContainerStarted","Data":"906b04fe2413ae5d2d6490d7529b20bd2e1c57a0144242177e65c8ec7758a16c"} Jul 22 03:06:53.521230 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:53.521193 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" event={"ID":"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a","Type":"ContainerStarted","Data":"8cb774b80bed0ca6e2a2c2e9bba910dbdbf54cf4b0e29348a215e3382b8a66be"} Jul 22 03:06:53.543924 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:53.543869 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" podStartSLOduration=1.543852238 podStartE2EDuration="1.543852238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-22 03:06:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:06:53.540709541 +0000 UTC m=+2855.318785515" watchObservedRunningTime="2026-07-22 03:06:53.543852238 +0000 UTC m=+2855.321928179" Jul 22 03:06:54.527582 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:54.527542 2570 generic.go:356] "Generic (PLEG): container finished" podID="aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" containerID="906b04fe2413ae5d2d6490d7529b20bd2e1c57a0144242177e65c8ec7758a16c" exitCode=0 Jul 22 03:06:54.527981 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:54.527630 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" event={"ID":"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a","Type":"ContainerDied","Data":"906b04fe2413ae5d2d6490d7529b20bd2e1c57a0144242177e65c8ec7758a16c"} Jul 22 03:06:55.663393 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:55.663371 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:06:55.803515 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:55.803396 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca\") pod \"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a\" (UID: \"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a\") " Jul 22 03:06:55.803781 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:55.803758 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" (UID: "aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:06:55.904804 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:55.904754 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a-maas-api-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:06:56.539619 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:56.539581 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" event={"ID":"aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a","Type":"ContainerDied","Data":"8cb774b80bed0ca6e2a2c2e9bba910dbdbf54cf4b0e29348a215e3382b8a66be"} Jul 22 03:06:56.539825 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:56.539636 2570 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8cb774b80bed0ca6e2a2c2e9bba910dbdbf54cf4b0e29348a215e3382b8a66be" Jul 22 03:06:56.539825 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:06:56.539595 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh" Jul 22 03:07:03.037072 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.037034 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:07:03.037567 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.037302 2570 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" podUID="136bfa59-5551-41e5-a587-e0804e089283" containerName="maas-api" containerID="cri-o://7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6" gracePeriod=30 Jul 22 03:07:03.294832 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.294768 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:07:03.370230 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.370199 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") pod \"136bfa59-5551-41e5-a587-e0804e089283\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " Jul 22 03:07:03.370418 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.370305 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s8n5r\" (UniqueName: \"kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r\") pod \"136bfa59-5551-41e5-a587-e0804e089283\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " Jul 22 03:07:03.370418 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.370331 2570 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca\") pod \"136bfa59-5551-41e5-a587-e0804e089283\" (UID: \"136bfa59-5551-41e5-a587-e0804e089283\") " Jul 22 03:07:03.370716 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.370694 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "136bfa59-5551-41e5-a587-e0804e089283" (UID: "136bfa59-5551-41e5-a587-e0804e089283"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 22 03:07:03.372390 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.372365 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "136bfa59-5551-41e5-a587-e0804e089283" (UID: "136bfa59-5551-41e5-a587-e0804e089283"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 22 03:07:03.372566 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.372548 2570 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r" (OuterVolumeSpecName: "kube-api-access-s8n5r") pod "136bfa59-5551-41e5-a587-e0804e089283" (UID: "136bfa59-5551-41e5-a587-e0804e089283"). InnerVolumeSpecName "kube-api-access-s8n5r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 22 03:07:03.471502 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.471458 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s8n5r\" (UniqueName: \"kubernetes.io/projected/136bfa59-5551-41e5-a587-e0804e089283-kube-api-access-s8n5r\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:07:03.471502 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.471494 2570 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/136bfa59-5551-41e5-a587-e0804e089283-openshift-service-ca\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:07:03.471502 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.471504 2570 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/136bfa59-5551-41e5-a587-e0804e089283-maas-api-tls\") on node \"ip-10-0-142-149.ec2.internal\" DevicePath \"\"" Jul 22 03:07:03.577867 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.577781 2570 generic.go:356] "Generic (PLEG): container finished" podID="136bfa59-5551-41e5-a587-e0804e089283" containerID="7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6" exitCode=0 Jul 22 03:07:03.577867 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.577832 2570 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" Jul 22 03:07:03.578046 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.577870 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" event={"ID":"136bfa59-5551-41e5-a587-e0804e089283","Type":"ContainerDied","Data":"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6"} Jul 22 03:07:03.578046 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.577926 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl" event={"ID":"136bfa59-5551-41e5-a587-e0804e089283","Type":"ContainerDied","Data":"9ffbbe06b06eadea9990d012dd5cdcf877bc6dcf7e8038cbb40b837584d95362"} Jul 22 03:07:03.578046 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.577956 2570 scope.go:117] "RemoveContainer" containerID="7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6" Jul 22 03:07:03.587588 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.587571 2570 scope.go:117] "RemoveContainer" containerID="7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6" Jul 22 03:07:03.587836 ip-10-0-142-149 kubenswrapper[2570]: E0722 03:07:03.587820 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6\": container with ID starting with 7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6 not found: ID does not exist" containerID="7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6" Jul 22 03:07:03.587886 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.587845 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6"} err="failed to get container status \"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6\": rpc error: code = NotFound desc = could not find container \"7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6\": container with ID starting with 7f91a5160800790092b8fd5aacb7b983ec7a9c2d62394fccd7e1c7489342d2d6 not found: ID does not exist" Jul 22 03:07:03.602747 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.602724 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:07:03.606607 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:03.606580 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-bb8c9d-6b44df867c-sjrbl"] Jul 22 03:07:04.748417 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:04.748385 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="136bfa59-5551-41e5-a587-e0804e089283" path="/var/lib/kubelet/pods/136bfa59-5551-41e5-a587-e0804e089283/volumes" Jul 22 03:07:07.667923 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:07.667888 2570 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh"] Jul 22 03:07:07.670321 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:07.670299 2570 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-bb8c9d-f76jh"] Jul 22 03:07:08.749013 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:08.748975 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" path="/var/lib/kubelet/pods/aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a/volumes" Jul 22 03:07:21.106648 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:21.106616 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-894c47d6f-rsp4q_0426d367-cac1-44df-a18d-a7176e5af680/manager/0.log" Jul 22 03:07:21.573090 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:21.573061 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-lqx45_ecddbba7-0af7-45c5-891d-5a2909871ced/copy-manifests/0.log" Jul 22 03:07:21.586723 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:21.586699 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-lqx45_ecddbba7-0af7-45c5-891d-5a2909871ced/manager/0.log" Jul 22 03:07:22.077811 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:22.077780 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-79f97668bb-v782h_fd4e218d-c483-4b50-84b0-7649b47f7314/maas-api/0.log" Jul 22 03:07:22.195267 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:22.195240 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744805-pg7qq_9ee59d10-e655-4ee7-8938-168c757496f3/cleanup/0.log" Jul 22 03:07:22.307025 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:22.306992 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29744820-wbqwz_188ffe3e-0974-4274-bcff-b16c50b30f6f/cleanup/0.log" Jul 22 03:07:22.417381 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:22.417299 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-7m2qg_e9815bb5-988b-400e-b9e9-30bd59757e58/postgres/0.log" Jul 22 03:07:23.203895 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.203869 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/pull/0.log" Jul 22 03:07:23.210189 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.210155 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/extract/0.log" Jul 22 03:07:23.216369 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.216352 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/util/0.log" Jul 22 03:07:23.328581 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.328551 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/util/0.log" Jul 22 03:07:23.338674 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.338648 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/pull/0.log" Jul 22 03:07:23.345709 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.345687 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/extract/0.log" Jul 22 03:07:23.460110 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.460028 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/util/0.log" Jul 22 03:07:23.470638 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.470617 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/pull/0.log" Jul 22 03:07:23.478856 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.478835 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/extract/0.log" Jul 22 03:07:23.597574 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.597549 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/util/0.log" Jul 22 03:07:23.604881 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.604859 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/pull/0.log" Jul 22 03:07:23.617767 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:23.617743 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/extract/0.log" Jul 22 03:07:25.092254 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:25.092221 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gkf5r_cc4e5add-200a-4613-8202-8a06e4bbbfca/discovery/0.log" Jul 22 03:07:25.552580 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:25.552555 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4zcd7_2d73cb06-f4df-4793-a376-a3f93ee60f9f/payload-pre-processing/0.log" Jul 22 03:07:25.666708 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:25.666678 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dd2b2_40a170f6-1f95-47bb-9b89-ff523bd57838/payload-processing/0.log" Jul 22 03:07:25.786123 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:25.786091 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d6ff7ddd6-r7kpn_ab3eb0fa-f69a-4497-a301-affe6659886c/router/0.log" Jul 22 03:07:26.272546 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:26.272514 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-qt2qb_bacded41-e287-44f9-bc1d-f9f79ec90525/main/0.log" Jul 22 03:07:26.502103 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:26.502073 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-85b68f5qctz_386df561-d012-4c66-8891-a17970772fb2/main/0.log" Jul 22 03:07:35.227999 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:35.227942 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-5bbrj_5f68fb46-7000-44a1-af02-354bd9b17c4a/global-pull-secret-syncer/0.log" Jul 22 03:07:35.431157 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:35.431086 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-f6xrz_52a3046d-e6a6-472d-a603-e6143318ce26/konnectivity-agent/0.log" Jul 22 03:07:35.492016 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:35.491941 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-149.ec2.internal_e7e85b7071d89f6d9d1647bbf823b423/haproxy/0.log" Jul 22 03:07:39.519357 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.519328 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/extract/0.log" Jul 22 03:07:39.543519 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.543484 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/util/0.log" Jul 22 03:07:39.562833 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.562804 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759nxsth_c99df528-8927-4321-989e-13bcfbe76134/pull/0.log" Jul 22 03:07:39.593372 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.593348 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/extract/0.log" Jul 22 03:07:39.613911 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.613874 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/util/0.log" Jul 22 03:07:39.632305 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.632281 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e04pvnv_0a33f8c1-112a-4035-898b-dab1d04d57fd/pull/0.log" Jul 22 03:07:39.655717 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.655689 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/extract/0.log" Jul 22 03:07:39.679817 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.679788 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/util/0.log" Jul 22 03:07:39.711074 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.711045 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73gssmr_5b89c749-870a-40fb-a56d-95f238ac322f/pull/0.log" Jul 22 03:07:39.761771 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.761745 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/extract/0.log" Jul 22 03:07:39.775694 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.775623 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/util/0.log" Jul 22 03:07:39.796190 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:39.796163 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1cn4bm_6ed129b2-0873-4730-8076-e421f83e89a1/pull/0.log" Jul 22 03:07:42.383218 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:42.383190 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-76cf8966f8-qmcb9_167c8564-82aa-4e98-8e1f-5314e4c84256/cluster-monitoring-operator/0.log" Jul 22 03:07:42.600386 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:42.600360 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmt6j_54d38012-2e57-4c53-a365-974e5d064832/node-exporter/0.log" Jul 22 03:07:42.622774 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:42.622751 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmt6j_54d38012-2e57-4c53-a365-974e5d064832/kube-rbac-proxy/0.log" Jul 22 03:07:42.640055 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:42.639988 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hmt6j_54d38012-2e57-4c53-a365-974e5d064832/init-textfile/0.log" Jul 22 03:07:45.521594 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.521548 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/2.log" Jul 22 03:07:45.526650 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.526634 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-565b7b569d-9qlx7_df65680e-1ec8-47cf-bbcd-c553bf9f81c2/console-operator/3.log" Jul 22 03:07:45.775338 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775240 2570 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2"] Jul 22 03:07:45.775669 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775651 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" containerName="revoke-keys" Jul 22 03:07:45.775751 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775673 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" containerName="revoke-keys" Jul 22 03:07:45.775751 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775707 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="136bfa59-5551-41e5-a587-e0804e089283" containerName="maas-api" Jul 22 03:07:45.775751 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775717 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="136bfa59-5551-41e5-a587-e0804e089283" containerName="maas-api" Jul 22 03:07:45.775901 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775804 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="136bfa59-5551-41e5-a587-e0804e089283" containerName="maas-api" Jul 22 03:07:45.775901 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.775821 2570 memory_manager.go:368] "RemoveStaleState removing state" podUID="aaa51c4a-e1ac-435d-8ff0-6a06a1d3548a" containerName="revoke-keys" Jul 22 03:07:45.778937 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.778918 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.781663 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.781626 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"openshift-service-ca.crt\"" Jul 22 03:07:45.782709 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.782686 2570 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-c69w5\"/\"default-dockercfg-hkft4\"" Jul 22 03:07:45.782817 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.782716 2570 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-c69w5\"/\"kube-root-ca.crt\"" Jul 22 03:07:45.789250 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.789224 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2"] Jul 22 03:07:45.855778 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.855749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-podres\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.855948 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.855791 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-proc\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.855948 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.855814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghbkf\" (UniqueName: \"kubernetes.io/projected/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-kube-api-access-ghbkf\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.855948 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.855841 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-sys\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.856062 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.855953 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-lib-modules\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956438 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956405 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-sys\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956472 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-lib-modules\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956500 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-podres\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956517 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-proc\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956536 2570 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghbkf\" (UniqueName: \"kubernetes.io/projected/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-kube-api-access-ghbkf\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956534 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-sys\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956627 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-proc\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956867 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956647 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-lib-modules\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.956867 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.956654 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-podres\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:45.965085 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:45.965061 2570 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghbkf\" (UniqueName: \"kubernetes.io/projected/c8ef0bfa-6e46-4962-9d5e-e8866af9d747-kube-api-access-ghbkf\") pod \"perf-node-gather-daemonset-j4bs2\" (UID: \"c8ef0bfa-6e46-4962-9d5e-e8866af9d747\") " pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:46.089347 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.089254 2570 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:46.106714 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.106689 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f59dbdcbc-f97f6_233cf869-8b3e-444f-a6b9-235e77ba927e/console/0.log" Jul 22 03:07:46.232693 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.232664 2570 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2"] Jul 22 03:07:46.234005 ip-10-0-142-149 kubenswrapper[2570]: W0722 03:07:46.233978 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podc8ef0bfa_6e46_4962_9d5e_e8866af9d747.slice/crio-310a1d1119bb4fba3a28e70f79059b5e6ab0fa14b5bcfaa33e56e38880db265c WatchSource:0}: Error finding container 310a1d1119bb4fba3a28e70f79059b5e6ab0fa14b5bcfaa33e56e38880db265c: Status 404 returned error can't find the container with id 310a1d1119bb4fba3a28e70f79059b5e6ab0fa14b5bcfaa33e56e38880db265c Jul 22 03:07:46.808839 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.808800 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" event={"ID":"c8ef0bfa-6e46-4962-9d5e-e8866af9d747","Type":"ContainerStarted","Data":"aa244383d42c65f4f630509733ef9f681f14c8f086fa9fd9467dcc496f5ccdbe"} Jul 22 03:07:46.809303 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.808861 2570 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" event={"ID":"c8ef0bfa-6e46-4962-9d5e-e8866af9d747","Type":"ContainerStarted","Data":"310a1d1119bb4fba3a28e70f79059b5e6ab0fa14b5bcfaa33e56e38880db265c"} Jul 22 03:07:46.809303 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.808903 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:46.829043 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.828995 2570 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" podStartSLOduration=1.828980696 podStartE2EDuration="1.828980696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-22 03:07:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-22 03:07:46.826707621 +0000 UTC m=+2908.604783560" watchObservedRunningTime="2026-07-22 03:07:46.828980696 +0000 UTC m=+2908.607056635" Jul 22 03:07:46.845625 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:46.845601 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78cf4c6f6-4nnf4_395d9965-3ac4-426f-9408-9f9d69424a12/volume-data-source-validator/0.log" Jul 22 03:07:47.899793 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:47.899762 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8dr7b_32b449dc-4ee5-49e3-8543-de214f550403/dns/0.log" Jul 22 03:07:47.919405 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:47.919381 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-8dr7b_32b449dc-4ee5-49e3-8543-de214f550403/kube-rbac-proxy/0.log" Jul 22 03:07:48.028007 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:48.027981 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-cqcd8_61f01f5c-b54c-4b97-8af8-e59860e23e78/dns-node-resolver/0.log" Jul 22 03:07:48.625763 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:48.625737 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gv24m_708a0732-481b-42fa-af89-ee23dfedff12/node-ca/0.log" Jul 22 03:07:49.706326 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:49.706300 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gkf5r_cc4e5add-200a-4613-8202-8a06e4bbbfca/discovery/0.log" Jul 22 03:07:49.837537 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:49.837513 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4zcd7_2d73cb06-f4df-4793-a376-a3f93ee60f9f/payload-pre-processing/0.log" Jul 22 03:07:49.863844 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:49.863816 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dd2b2_40a170f6-1f95-47bb-9b89-ff523bd57838/payload-processing/0.log" Jul 22 03:07:49.882461 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:49.882436 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d6ff7ddd6-r7kpn_ab3eb0fa-f69a-4497-a301-affe6659886c/router/0.log" Jul 22 03:07:50.552880 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:50.552854 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4jq5p_bdae54c5-819f-46d8-80e4-0160b8ae9732/serve-healthcheck-canary/0.log" Jul 22 03:07:51.033592 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:51.033558 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-l4vnz_fc82755c-0dce-45a5-bd49-a787c95dc17e/insights-operator/0.log" Jul 22 03:07:51.034591 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:51.034576 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-555d9d7885-l4vnz_fc82755c-0dce-45a5-bd49-a787c95dc17e/insights-operator/1.log" Jul 22 03:07:51.178822 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:51.178795 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2m8q_12307f20-b8c0-433a-86a9-64ad31c18fcd/kube-rbac-proxy/0.log" Jul 22 03:07:51.197394 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:51.197372 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2m8q_12307f20-b8c0-433a-86a9-64ad31c18fcd/exporter/0.log" Jul 22 03:07:51.214867 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:51.214835 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-s2m8q_12307f20-b8c0-433a-86a9-64ad31c18fcd/extractor/0.log" Jul 22 03:07:52.825081 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:52.825053 2570 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-c69w5/perf-node-gather-daemonset-j4bs2" Jul 22 03:07:53.613545 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:53.613507 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-894c47d6f-rsp4q_0426d367-cac1-44df-a18d-a7176e5af680/manager/0.log" Jul 22 03:07:53.808192 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:53.808156 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-lqx45_ecddbba7-0af7-45c5-891d-5a2909871ced/manager/0.log" Jul 22 03:07:53.823142 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:53.823112 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7cb9fd785c-lqx45_ecddbba7-0af7-45c5-891d-5a2909871ced/copy-manifests/0.log" Jul 22 03:07:55.238798 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:55.238763 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-645b87676-2t2wm_569d85ac-02cd-4800-b584-ffb1b5e0a460/manager/0.log" Jul 22 03:07:56.268248 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:56.268214 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gkf5r_cc4e5add-200a-4613-8202-8a06e4bbbfca/discovery/0.log" Jul 22 03:07:56.356810 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:56.356781 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4zcd7_2d73cb06-f4df-4793-a376-a3f93ee60f9f/payload-pre-processing/0.log" Jul 22 03:07:56.379229 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:56.379201 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dd2b2_40a170f6-1f95-47bb-9b89-ff523bd57838/payload-processing/0.log" Jul 22 03:07:56.396802 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:56.396765 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d6ff7ddd6-r7kpn_ab3eb0fa-f69a-4497-a301-affe6659886c/router/0.log" Jul 22 03:07:57.733417 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:57.733382 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-gkf5r_cc4e5add-200a-4613-8202-8a06e4bbbfca/discovery/0.log" Jul 22 03:07:57.826643 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:57.826612 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-4zcd7_2d73cb06-f4df-4793-a376-a3f93ee60f9f/payload-pre-processing/0.log" Jul 22 03:07:57.846314 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:57.846278 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-dd2b2_40a170f6-1f95-47bb-9b89-ff523bd57838/payload-processing/0.log" Jul 22 03:07:57.858840 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:07:57.858819 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-6d6ff7ddd6-r7kpn_ab3eb0fa-f69a-4497-a301-affe6659886c/router/0.log" Jul 22 03:08:00.721987 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:00.721952 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-bf5jw_0934c3d1-503f-49e4-8a19-235af2f466fb/kube-storage-version-migrator-operator/1.log" Jul 22 03:08:00.723151 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:00.723111 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-7658d9b749-bf5jw_0934c3d1-503f-49e4-8a19-235af2f466fb/kube-storage-version-migrator-operator/0.log" Jul 22 03:08:02.100808 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.100783 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/kube-multus-additional-cni-plugins/0.log" Jul 22 03:08:02.126875 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.126850 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/egress-router-binary-copy/0.log" Jul 22 03:08:02.149270 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.149246 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/cni-plugins/0.log" Jul 22 03:08:02.170984 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.170955 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/bond-cni-plugin/0.log" Jul 22 03:08:02.197691 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.197666 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/routeoverride-cni/0.log" Jul 22 03:08:02.236350 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.236326 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/whereabouts-cni-bincopy/0.log" Jul 22 03:08:02.264569 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.264539 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7zhjr_bcc7da07-3195-4440-8bd5-68f8b8684573/whereabouts-cni/0.log" Jul 22 03:08:02.514466 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.514389 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mj7l7_17b9d2f8-6567-4147-87fd-f8ef4d9f4cc2/kube-multus/0.log" Jul 22 03:08:02.614003 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.613973 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-944xj_4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f/network-metrics-daemon/0.log" Jul 22 03:08:02.636191 ip-10-0-142-149 kubenswrapper[2570]: I0722 03:08:02.636163 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-944xj_4ba9aa5d-a6c6-42f0-b4d4-0a61d8612f2f/kube-rbac-proxy/0.log"