Jul 23 00:20:08.590168 ip-10-0-133-87 systemd[1]: Starting Kubernetes Kubelet... Jul 23 00:20:08.959884 ip-10-0-133-87 kubenswrapper[2575]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 00:20:08.959884 ip-10-0-133-87 kubenswrapper[2575]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 23 00:20:08.959884 ip-10-0-133-87 kubenswrapper[2575]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 00:20:08.959884 ip-10-0-133-87 kubenswrapper[2575]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 23 00:20:08.959884 ip-10-0-133-87 kubenswrapper[2575]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 23 00:20:08.960729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.960650 2575 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 23 00:20:08.965712 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965691 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.965712 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965708 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.965712 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965712 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965718 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965724 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965727 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965730 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965733 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965736 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965739 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965742 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965744 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965747 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965750 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965752 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965755 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965757 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965760 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965762 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965765 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965767 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965770 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.965877 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965772 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965775 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965778 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965781 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965784 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965786 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965795 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965798 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965800 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965803 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965806 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965808 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965811 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965813 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965816 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965819 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965822 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965840 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965850 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965852 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.966347 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965855 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965858 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965860 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965863 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965867 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965869 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965872 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965875 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965877 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965881 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965883 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965886 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965889 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965891 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965894 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965897 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965899 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965902 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965904 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.966837 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965906 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965909 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965911 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965914 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965917 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965919 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965922 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965926 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965928 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965931 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965934 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965936 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965939 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965941 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965945 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965949 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965952 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965955 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965957 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965960 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.967285 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965963 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965966 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965970 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965973 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965975 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965978 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965980 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965983 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965985 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965988 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965991 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965993 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965996 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.965998 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966001 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966003 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966007 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966011 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966764 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.967796 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966771 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966776 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966779 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966782 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966785 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966788 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966790 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966793 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966795 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966798 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966800 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966803 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966806 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966808 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966811 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966813 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966816 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966819 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966822 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966824 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.968260 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966842 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966847 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966851 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966854 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966857 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966861 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966864 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966866 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966869 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966871 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966875 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966877 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966880 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966883 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966886 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966888 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966891 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966893 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966896 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.968741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966899 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966902 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966904 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966908 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966912 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966914 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966917 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966920 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966923 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966925 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966929 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966932 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966934 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966937 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966940 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966943 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966946 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966949 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966951 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.969230 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966955 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966957 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966960 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966962 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966965 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966968 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966970 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966973 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966976 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966978 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966981 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966983 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966986 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966988 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966991 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966993 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966996 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.966998 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967001 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967003 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.969689 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967005 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967008 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967011 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967013 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967017 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967019 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967022 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967026 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967029 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967032 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967035 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967037 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967040 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967042 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967045 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967047 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967050 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967053 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967056 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967058 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.970178 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967443 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967448 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967451 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967454 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967457 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967459 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967462 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967464 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967467 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967470 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967472 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967475 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967477 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967480 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967483 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967486 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967490 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967492 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967495 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967498 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.970647 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967501 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967503 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967506 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967509 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967513 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967516 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967519 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967521 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967524 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967527 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967529 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967532 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967534 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967537 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967539 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967541 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967544 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967547 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967549 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.971153 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967552 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967554 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967557 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967559 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967562 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967564 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967567 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967569 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967572 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967575 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967577 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967580 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967582 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967585 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967587 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967590 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967593 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967595 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967598 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967600 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.971599 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967603 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967605 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967609 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967612 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967615 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967618 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967622 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967626 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967628 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967631 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967634 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967637 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967639 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967642 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967646 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967648 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967650 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967653 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967655 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.972107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967658 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967661 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967663 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967666 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967669 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967671 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967673 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967676 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967678 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967684 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967686 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967689 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967691 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967694 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967696 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967699 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967701 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967703 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967706 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967709 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.972552 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.967711 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967779 2575 flags.go:64] FLAG: --address="0.0.0.0" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967785 2575 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967792 2575 flags.go:64] FLAG: --anonymous-auth="true" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967799 2575 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967806 2575 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967811 2575 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967817 2575 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967823 2575 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967838 2575 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967841 2575 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967845 2575 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967848 2575 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967852 2575 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967855 2575 flags.go:64] FLAG: --cgroup-root="" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967858 2575 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967861 2575 flags.go:64] FLAG: --client-ca-file="" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967865 2575 flags.go:64] FLAG: --cloud-provider="external" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967867 2575 flags.go:64] FLAG: --cluster-dns="[]" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967871 2575 flags.go:64] FLAG: --cluster-domain="" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967874 2575 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967879 2575 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967883 2575 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967886 2575 flags.go:64] FLAG: --container-log-max-files="5" Jul 23 00:20:08.973035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967890 2575 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967893 2575 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967896 2575 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967899 2575 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967902 2575 flags.go:64] FLAG: --contention-profiling="false" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967905 2575 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967908 2575 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967911 2575 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967914 2575 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967918 2575 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967921 2575 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967924 2575 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967926 2575 flags.go:64] FLAG: --enable-load-reader="false" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967929 2575 flags.go:64] FLAG: --enable-server="true" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967932 2575 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967936 2575 flags.go:64] FLAG: --event-burst="100" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967939 2575 flags.go:64] FLAG: --event-qps="50" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967942 2575 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967945 2575 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967948 2575 flags.go:64] FLAG: --eviction-hard="" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967952 2575 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967955 2575 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967958 2575 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967960 2575 flags.go:64] FLAG: --eviction-soft="" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967963 2575 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 23 00:20:08.973620 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967966 2575 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967969 2575 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967972 2575 flags.go:64] FLAG: --experimental-mounter-path="" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967975 2575 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967979 2575 flags.go:64] FLAG: --fail-swap-on="true" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967982 2575 flags.go:64] FLAG: --feature-gates="" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967986 2575 flags.go:64] FLAG: --file-check-frequency="20s" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967989 2575 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967992 2575 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967995 2575 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.967997 2575 flags.go:64] FLAG: --healthz-port="10248" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968001 2575 flags.go:64] FLAG: --help="false" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968003 2575 flags.go:64] FLAG: --hostname-override="ip-10-0-133-87.ec2.internal" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968007 2575 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968009 2575 flags.go:64] FLAG: --http-check-frequency="20s" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968012 2575 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968015 2575 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968018 2575 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968021 2575 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968024 2575 flags.go:64] FLAG: --image-service-endpoint="" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968027 2575 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968030 2575 flags.go:64] FLAG: --kube-api-burst="100" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968032 2575 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968036 2575 flags.go:64] FLAG: --kube-api-qps="50" Jul 23 00:20:08.974229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968038 2575 flags.go:64] FLAG: --kube-reserved="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968041 2575 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968044 2575 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968047 2575 flags.go:64] FLAG: --kubelet-cgroups="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968052 2575 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968054 2575 flags.go:64] FLAG: --lock-file="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968057 2575 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968060 2575 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968063 2575 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968068 2575 flags.go:64] FLAG: --log-json-split-stream="false" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968071 2575 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968074 2575 flags.go:64] FLAG: --log-text-split-stream="false" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968078 2575 flags.go:64] FLAG: --logging-format="text" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968081 2575 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968084 2575 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968087 2575 flags.go:64] FLAG: --manifest-url="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968090 2575 flags.go:64] FLAG: --manifest-url-header="" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968094 2575 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968097 2575 flags.go:64] FLAG: --max-open-files="1000000" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968101 2575 flags.go:64] FLAG: --max-pods="110" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968104 2575 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968107 2575 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968110 2575 flags.go:64] FLAG: --memory-manager-policy="None" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968112 2575 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968115 2575 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 23 00:20:08.974803 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968118 2575 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968121 2575 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968128 2575 flags.go:64] FLAG: --node-status-max-images="50" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968131 2575 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968134 2575 flags.go:64] FLAG: --oom-score-adj="-999" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968137 2575 flags.go:64] FLAG: --pod-cidr="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968140 2575 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968144 2575 flags.go:64] FLAG: --pod-manifest-path="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968147 2575 flags.go:64] FLAG: --pod-max-pids="-1" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968150 2575 flags.go:64] FLAG: --pods-per-core="0" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968153 2575 flags.go:64] FLAG: --port="10250" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968157 2575 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968160 2575 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0d9b4755bf53aa116" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968163 2575 flags.go:64] FLAG: --qos-reserved="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968166 2575 flags.go:64] FLAG: --read-only-port="10255" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968169 2575 flags.go:64] FLAG: --register-node="true" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968172 2575 flags.go:64] FLAG: --register-with-taints="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968175 2575 flags.go:64] FLAG: --registry-burst="10" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968179 2575 flags.go:64] FLAG: --registry-qps="5" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968182 2575 flags.go:64] FLAG: --reserved-cpus="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968186 2575 flags.go:64] FLAG: --reserved-memory="" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968189 2575 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968192 2575 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968195 2575 flags.go:64] FLAG: --rotate-certificates="false" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968198 2575 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 23 00:20:08.975430 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968201 2575 flags.go:64] FLAG: --runonce="false" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968203 2575 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968206 2575 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968209 2575 flags.go:64] FLAG: --seccomp-default="false" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968212 2575 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968215 2575 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968218 2575 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968221 2575 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968224 2575 flags.go:64] FLAG: --storage-driver-password="root" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968227 2575 flags.go:64] FLAG: --storage-driver-secure="false" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968230 2575 flags.go:64] FLAG: --storage-driver-table="stats" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968233 2575 flags.go:64] FLAG: --storage-driver-user="root" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968235 2575 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968238 2575 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968241 2575 flags.go:64] FLAG: --system-cgroups="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968244 2575 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968249 2575 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968252 2575 flags.go:64] FLAG: --tls-cert-file="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968255 2575 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968259 2575 flags.go:64] FLAG: --tls-min-version="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968262 2575 flags.go:64] FLAG: --tls-private-key-file="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968265 2575 flags.go:64] FLAG: --topology-manager-policy="none" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968268 2575 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968271 2575 flags.go:64] FLAG: --topology-manager-scope="container" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968274 2575 flags.go:64] FLAG: --v="2" Jul 23 00:20:08.976035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968278 2575 flags.go:64] FLAG: --version="false" Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968283 2575 flags.go:64] FLAG: --vmodule="" Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968288 2575 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.968292 2575 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968420 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968424 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968427 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968430 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968433 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968436 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968439 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968442 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968444 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968446 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968449 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968451 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968454 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968456 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968459 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968462 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.977006 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968464 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968468 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968472 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968475 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968478 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968482 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968485 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968487 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968490 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968493 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968496 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968500 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968503 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968506 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968511 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968513 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968516 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968519 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968521 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.977911 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968524 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968526 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968529 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968531 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968533 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968536 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968539 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968541 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968544 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968546 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968549 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968551 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968554 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968556 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968559 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968561 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968564 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968566 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968571 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.978704 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968573 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968576 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968578 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968581 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968583 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968586 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968588 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968591 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968595 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968601 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968603 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968606 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968609 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968611 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968613 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968616 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968619 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968621 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968624 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.979521 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968626 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968629 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968631 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968647 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968650 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968654 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968658 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968661 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968664 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968666 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968669 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968671 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968676 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968679 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968681 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968684 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968686 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968689 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968691 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968694 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.980384 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968696 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968700 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968703 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968709 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968712 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.968715 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.970584 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.979132 2575 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.979155 2575 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.979174 2575 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.979184 2575 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979290 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979299 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979304 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979309 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.981023 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979314 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979318 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979323 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979327 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979332 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979336 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979340 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979344 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979348 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979353 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979358 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979362 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979366 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979370 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979375 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979379 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979383 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979387 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979392 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979396 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.981583 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979400 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979405 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979409 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979413 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979417 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979423 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979428 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979432 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979436 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979441 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979445 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979449 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979453 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979457 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979462 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979466 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979470 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979474 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979478 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979482 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.982436 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979487 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979491 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979495 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979500 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979504 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979508 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979512 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979516 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979520 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979524 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979528 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979532 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979536 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979542 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979547 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979553 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979558 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979564 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979570 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979574 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.983022 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979578 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979583 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979587 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979591 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979595 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979600 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979604 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979608 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979612 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979617 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979621 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979625 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979629 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979634 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979638 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979644 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979650 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979655 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979659 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.983606 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979664 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979668 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979673 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979677 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979683 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979689 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979693 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979697 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979701 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979706 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979710 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979714 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979719 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979723 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979727 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.984107 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979731 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.979739 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979952 2575 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979961 2575 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979967 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979972 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979977 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979982 2575 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979986 2575 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979991 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979995 2575 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.979999 2575 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980003 2575 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980008 2575 feature_gate.go:328] unrecognized feature gate: Example2 Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980012 2575 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 23 00:20:08.984468 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980016 2575 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980021 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980025 2575 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980029 2575 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980033 2575 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980037 2575 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980041 2575 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980046 2575 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980050 2575 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980055 2575 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980059 2575 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980064 2575 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980068 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980072 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980076 2575 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980081 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980085 2575 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980090 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980094 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980098 2575 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 23 00:20:08.985069 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980102 2575 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980107 2575 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980111 2575 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980115 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980119 2575 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980124 2575 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980128 2575 feature_gate.go:328] unrecognized feature gate: Example Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980132 2575 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980136 2575 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980140 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980144 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980149 2575 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980153 2575 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980157 2575 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980161 2575 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980167 2575 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980174 2575 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980178 2575 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980183 2575 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980187 2575 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 23 00:20:08.985609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980192 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980196 2575 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980201 2575 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980206 2575 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980210 2575 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980215 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980219 2575 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980224 2575 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980229 2575 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980233 2575 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980237 2575 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980242 2575 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980246 2575 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980250 2575 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980254 2575 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980258 2575 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980262 2575 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980268 2575 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980273 2575 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980277 2575 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 23 00:20:08.986119 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980281 2575 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980285 2575 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980289 2575 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980294 2575 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980298 2575 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980302 2575 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980307 2575 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980312 2575 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980316 2575 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980320 2575 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980324 2575 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980328 2575 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980332 2575 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980339 2575 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980344 2575 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980350 2575 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980355 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980359 2575 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980364 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 23 00:20:08.986730 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980368 2575 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980372 2575 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980377 2575 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980381 2575 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980385 2575 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980389 2575 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:08.980393 2575 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.980401 2575 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.980548 2575 server.go:962] "Client rotation is on, will bootstrap in background" Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.983163 2575 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.984525 2575 server.go:1019] "Starting client certificate rotation" Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.984631 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.984690 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 23 00:20:08.987493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.987076 2575 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 00:20:08.990524 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:08.990503 2575 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 23 00:20:09.000876 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.000858 2575 log.go:25] "Validated CRI v1 runtime API" Jul 23 00:20:09.002921 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.002906 2575 log.go:25] "Validated CRI v1 image API" Jul 23 00:20:09.003768 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.003749 2575 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 23 00:20:09.007173 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.007157 2575 fs.go:135] Filesystem UUIDs: map[3264d6da-28a8-47b3-9983-4e70d73943ac:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 a756a58a-8ae5-4baa-802f-97d4de566336:/dev/nvme0n1p4] Jul 23 00:20:09.007244 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.007173 2575 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 23 00:20:09.011261 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.011242 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 00:20:09.012293 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.012192 2575 manager.go:216] Machine: {Timestamp:2026-07-23 00:20:09.010937291 +0000 UTC m=+0.317085276 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099183 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec296498d836ded0d4e84a35d5cfd9df SystemUUID:ec296498-d836-ded0-d4e8-4a35d5cfd9df BootID:68485527-d665-4885-9ced-0af4e74b1882 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:da:0b:76:34:8b Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:da:0b:76:34:8b Speed:0 Mtu:9001} {Name:ovs-system MacAddress:aa:cf:45:8a:ce:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 23 00:20:09.012293 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.012290 2575 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 23 00:20:09.012418 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.012358 2575 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 23 00:20:09.015148 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015124 2575 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 23 00:20:09.015271 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015151 2575 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-87.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 23 00:20:09.015725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015716 2575 topology_manager.go:138] "Creating topology manager with none policy" Jul 23 00:20:09.015754 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015727 2575 container_manager_linux.go:309] "Creating device plugin manager" Jul 23 00:20:09.015754 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015734 2575 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 00:20:09.015754 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.015746 2575 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 23 00:20:09.016997 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.016987 2575 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 23 00:20:09.019731 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019720 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 23 00:20:09.019860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019850 2575 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 23 00:20:09.019904 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019878 2575 kubelet.go:486] "Attempting to sync node with API server" Jul 23 00:20:09.019904 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019887 2575 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 23 00:20:09.019904 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019901 2575 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 23 00:20:09.020005 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019909 2575 kubelet.go:398] "Adding apiserver pod source" Jul 23 00:20:09.020005 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.019918 2575 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 23 00:20:09.020212 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.020200 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 00:20:09.020252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.020220 2575 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 23 00:20:09.023047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.023030 2575 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 23 00:20:09.024421 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.024407 2575 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 23 00:20:09.024479 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.024438 2575 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 23 00:20:09.025086 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025069 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 23 00:20:09.025086 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025087 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 23 00:20:09.025220 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025094 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 23 00:20:09.025220 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025100 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 23 00:20:09.025220 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025108 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 23 00:20:09.025220 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025113 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025231 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025236 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025243 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025248 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025263 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 23 00:20:09.025327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025271 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 23 00:20:09.025938 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025929 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 23 00:20:09.025938 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.025938 2575 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 23 00:20:09.028871 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.028855 2575 server.go:1268] "Started kubelet" Jul 23 00:20:09.029020 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.028939 2575 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 23 00:20:09.029546 ip-10-0-133-87 systemd[1]: Started Kubernetes Kubelet. Jul 23 00:20:09.029717 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.029663 2575 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 23 00:20:09.032989 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.032967 2575 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 23 00:20:09.033078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.033055 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8xsxs" Jul 23 00:20:09.033633 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.033609 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-87.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 23 00:20:09.033633 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.033610 2575 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 23 00:20:09.033799 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.033783 2575 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-87.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 23 00:20:09.035448 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.034735 2575 server.go:310] "Adding debug handlers to kubelet server" Jul 23 00:20:09.035863 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.035793 2575 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 23 00:20:09.037311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.037292 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8xsxs" Jul 23 00:20:09.039068 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.039050 2575 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 23 00:20:09.039692 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.037976 2575 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-87.ec2.internal.18c4c33aa75abc2c default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-87.ec2.internal,UID:ip-10-0-133-87.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-87.ec2.internal,},FirstTimestamp:2026-07-23 00:20:09.028820012 +0000 UTC m=+0.334967999,LastTimestamp:2026-07-23 00:20:09.028820012 +0000 UTC m=+0.334967999,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-87.ec2.internal,}" Jul 23 00:20:09.043341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.043325 2575 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 23 00:20:09.043341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.043333 2575 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 23 00:20:09.043475 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.043387 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.044277 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044261 2575 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 23 00:20:09.044277 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044264 2575 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 23 00:20:09.044397 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044286 2575 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 23 00:20:09.044446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044397 2575 reconstruct.go:97] "Volume reconstruction finished" Jul 23 00:20:09.044446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044406 2575 reconciler.go:29] "Reconciler: start to sync state" Jul 23 00:20:09.044660 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044644 2575 factory.go:153] Registering CRI-O factory Jul 23 00:20:09.044660 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044659 2575 factory.go:223] Registration of the crio container factory successfully Jul 23 00:20:09.044782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044702 2575 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 23 00:20:09.044782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044709 2575 factory.go:55] Registering systemd factory Jul 23 00:20:09.044782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044714 2575 factory.go:223] Registration of the systemd container factory successfully Jul 23 00:20:09.044782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044729 2575 factory.go:103] Registering Raw factory Jul 23 00:20:09.044782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.044738 2575 manager.go:1196] Started watching for new ooms in manager Jul 23 00:20:09.045154 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.045138 2575 manager.go:349] Starting recovery of all containers Jul 23 00:20:09.055237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.055214 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:09.056558 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.056542 2575 manager.go:354] Recovery completed Jul 23 00:20:09.058036 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.058017 2575 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-87.ec2.internal\" not found" node="ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.060458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.060445 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.062202 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062185 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.062265 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062218 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.062265 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062235 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.062703 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062689 2575 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 23 00:20:09.062773 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062704 2575 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 23 00:20:09.062773 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062724 2575 state_mem.go:36] "Initialized new in-memory state store" Jul 23 00:20:09.062773 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.062696 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.063436 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063421 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.063506 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063449 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.063506 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063459 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.063882 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063864 2575 policy_none.go:49] "None policy: Start" Jul 23 00:20:09.063882 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063882 2575 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 23 00:20:09.064017 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.063895 2575 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 23 00:20:09.064971 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.064959 2575 policy_none.go:47] "Start" Jul 23 00:20:09.101437 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101316 2575 manager.go:337] "Starting Device Plugin manager" Jul 23 00:20:09.101512 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.101454 2575 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 23 00:20:09.101512 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101466 2575 server.go:87] "Starting device plugin registration server" Jul 23 00:20:09.101690 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101677 2575 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 23 00:20:09.101748 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101692 2575 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 23 00:20:09.101796 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101785 2575 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 23 00:20:09.101901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101879 2575 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 23 00:20:09.101901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.101892 2575 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 23 00:20:09.102350 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.102329 2575 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 23 00:20:09.102420 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.102388 2575 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.202586 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.202560 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.203441 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.203423 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.203515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.203453 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.203515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.203467 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.203515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.203492 2575 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.210091 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.210050 2575 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.210091 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.210071 2575 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-87.ec2.internal\": node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.223143 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.223118 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.228924 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.228897 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 23 00:20:09.230120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.230103 2575 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 23 00:20:09.230206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.230151 2575 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 23 00:20:09.230206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.230176 2575 kubelet.go:2448] "Starting kubelet main sync loop" Jul 23 00:20:09.230319 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.230214 2575 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 23 00:20:09.232589 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.232572 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:09.326674 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.326624 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.331117 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.331100 2575 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal"] Jul 23 00:20:09.331172 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.331164 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.332870 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.332853 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.332934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.332886 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.332934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.332896 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.334271 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.334248 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.334403 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.334376 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.334445 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.334423 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.335089 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335073 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.335160 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335089 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.335160 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335104 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.335160 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335114 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.335160 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335121 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.335160 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.335125 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.336326 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.336309 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.336380 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.336344 2575 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 23 00:20:09.337095 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.337079 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientMemory" Jul 23 00:20:09.337186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.337106 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasNoDiskPressure" Jul 23 00:20:09.337186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.337121 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeHasSufficientPID" Jul 23 00:20:09.347018 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.346998 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.347105 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.347026 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.347105 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.347044 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9ca0bc95fbca063e66eb3b93e200e1c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"a9ca0bc95fbca063e66eb3b93e200e1c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.364677 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.364658 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-87.ec2.internal\" not found" node="ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.368346 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.368320 2575 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-87.ec2.internal\" not found" node="ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.427477 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.427454 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.447929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.447910 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.448003 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.447932 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.448003 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.447990 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9ca0bc95fbca063e66eb3b93e200e1c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"a9ca0bc95fbca063e66eb3b93e200e1c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.448102 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.448038 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.448102 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.448044 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/83586bb93cbfd50d0fdc79ded33ba2b5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal\" (UID: \"83586bb93cbfd50d0fdc79ded33ba2b5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.448102 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.447950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a9ca0bc95fbca063e66eb3b93e200e1c-config\") pod \"kube-apiserver-proxy-ip-10-0-133-87.ec2.internal\" (UID: \"a9ca0bc95fbca063e66eb3b93e200e1c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.528463 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.528410 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.634949 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.634915 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.666717 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.666692 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.671884 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.671863 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:09.735918 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.735876 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.839136 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.839062 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.939363 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:09.939325 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:09.984141 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.984120 2575 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 23 00:20:09.984558 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.984272 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 00:20:09.984558 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:09.984276 2575 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 23 00:20:10.039682 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:10.039654 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:10.042297 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.042269 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-22 00:15:09 +0000 UTC" deadline="2028-04-23 19:07:05.621268566 +0000 UTC" Jul 23 00:20:10.042297 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.042298 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15378h46m55.578974059s" Jul 23 00:20:10.140552 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:10.140491 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:10.191245 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.191225 2575 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:10.241034 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:10.241009 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:10.341931 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:10.341906 2575 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-87.ec2.internal\" not found" Jul 23 00:20:10.347072 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:10.347037 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9ca0bc95fbca063e66eb3b93e200e1c.slice/crio-daa4dfbce6fd860bdcf4a307f4b99865a52970083a3840d9ade1b4aa86d69d6d WatchSource:0}: Error finding container daa4dfbce6fd860bdcf4a307f4b99865a52970083a3840d9ade1b4aa86d69d6d: Status 404 returned error can't find the container with id daa4dfbce6fd860bdcf4a307f4b99865a52970083a3840d9ade1b4aa86d69d6d Jul 23 00:20:10.355682 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.355663 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:20:10.359990 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:10.359970 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83586bb93cbfd50d0fdc79ded33ba2b5.slice/crio-857addb30fcfc47db1a5f0e26d027119e16fd23641f681171e77e830eace21d3 WatchSource:0}: Error finding container 857addb30fcfc47db1a5f0e26d027119e16fd23641f681171e77e830eace21d3: Status 404 returned error can't find the container with id 857addb30fcfc47db1a5f0e26d027119e16fd23641f681171e77e830eace21d3 Jul 23 00:20:10.389075 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.389050 2575 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:10.443641 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.443565 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" Jul 23 00:20:10.456604 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.456576 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 00:20:10.457818 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.457802 2575 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" Jul 23 00:20:10.466073 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.466054 2575 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 23 00:20:10.879946 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:10.879860 2575 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:11.020974 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.020933 2575 apiserver.go:52] "Watching apiserver" Jul 23 00:20:11.032775 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.031825 2575 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 23 00:20:11.032972 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.032827 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-ldjk4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal","openshift-multus/multus-additional-cni-plugins-25s8z","openshift-network-operator/iptables-alerter-cxfpx","openshift-ovn-kubernetes/ovnkube-node-fhw25","kube-system/konnectivity-agent-ztl74","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9","openshift-cluster-node-tuning-operator/tuned-2pxw5","openshift-image-registry/node-ca-74sfp","openshift-multus/multus-q2rrf","openshift-multus/network-metrics-daemon-krcjb","openshift-network-diagnostics/network-check-target-945mz","kube-system/global-pull-secret-syncer-htfz9","kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal"] Jul 23 00:20:11.035263 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.035236 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.036982 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.036958 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.037102 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.037050 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:11.038072 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.038031 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.038193 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.038088 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.038193 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.038168 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 23 00:20:11.038314 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.038184 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2kst5\"" Jul 23 00:20:11.038391 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.038366 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.039715 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.039694 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.040867 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.040825 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 23 00:20:11.041281 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041258 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 23 00:20:11.041407 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041281 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.041407 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041344 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.041672 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041653 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-chzrv\"" Jul 23 00:20:11.041814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041792 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.041945 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.041814 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 23 00:20:11.042867 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.042835 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 23 00:20:11.042968 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.042935 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-x6dgg\"" Jul 23 00:20:11.043163 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.043051 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.043163 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.043111 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.043398 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.043217 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.043527 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.043481 2575 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 23 00:20:11.044224 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.043992 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.044224 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.044020 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 23 00:20:11.044366 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.044325 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.044784 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.044532 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.045321 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045267 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 23 00:20:11.045654 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045320 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-vp6lw\"" Jul 23 00:20:11.045654 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045388 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 23 00:20:11.045654 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045371 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 23 00:20:11.046027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045853 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-mrvtq\"" Jul 23 00:20:11.046027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.045912 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 23 00:20:11.046157 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.046140 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 23 00:20:11.046221 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.046208 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.047026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.046955 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.047026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.047013 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.047175 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.047076 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nf2lf\"" Jul 23 00:20:11.047175 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.047118 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 23 00:20:11.047667 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.047389 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.048402 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.048381 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-dvbr9\"" Jul 23 00:20:11.049354 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.048749 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.049354 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.048785 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.049354 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.049003 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.050118 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.050070 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 23 00:20:11.050226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.050158 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-krwrk\"" Jul 23 00:20:11.050226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.050205 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 23 00:20:11.050420 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.050405 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.050504 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.050480 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:11.051347 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.051320 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 23 00:20:11.051347 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.051320 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-f7lv7\"" Jul 23 00:20:11.052556 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.052536 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:11.052655 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.052606 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:11.054956 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.054937 2575 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 23 00:20:11.057226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.057329 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-78z4p\" (UniqueName: \"kubernetes.io/projected/912ac498-7d64-46a8-bc09-5cb60db5ea55-kube-api-access-78z4p\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.057329 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-serviceca\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.057329 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057316 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-var-lib-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057341 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-ovn\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057365 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-bin\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057397 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057421 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97511f57-175e-4342-8d95-5abd34d00543-host-slash\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.057461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057442 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vq7zb\" (UniqueName: \"kubernetes.io/projected/97511f57-175e-4342-8d95-5abd34d00543-kube-api-access-vq7zb\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057487 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovn-node-metrics-cert\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057510 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-script-lib\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057537 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057581 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057605 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/97511f57-175e-4342-8d95-5abd34d00543-iptables-alerter-script\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057624 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-etc-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.057662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057649 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-agent-certs\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.058133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057672 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-konnectivity-ca\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.058133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057701 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-registration-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.058133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057738 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-etc-selinux\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.058133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057768 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22prn\" (UniqueName: \"kubernetes.io/projected/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-kube-api-access-22prn\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.058133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.057796 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-systemd-units\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.058419 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.058398 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-netns\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.058480 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.058458 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-node-log\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.058982 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.058821 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-env-overrides\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.059089 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059024 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt4c5\" (UniqueName: \"kubernetes.io/projected/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-kube-api-access-tt4c5\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.059089 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059063 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-device-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.059207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059097 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-cnibin\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.059207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059130 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-binary-copy\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.059207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059162 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-kubelet\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.059207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059195 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-kubelet-config\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.059412 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059228 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-os-release\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.059412 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059306 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-host\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.059412 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059366 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-log-socket\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.059559 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-sys-fs\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.059619 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059565 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7zmpm\" (UniqueName: \"kubernetes.io/projected/a6650f62-d7b7-41ef-b541-b642dac6d011-kube-api-access-7zmpm\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.059619 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059607 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-dbus\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.059823 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059760 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-system-cni-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.060026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059827 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.060026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-slash\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.060026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059943 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.060026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.059978 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-socket-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.060026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.060010 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.060264 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.060048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-systemd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.060264 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.060220 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-netd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.060264 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.060257 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-config\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.072491 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.072464 2575 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-8jwcl" Jul 23 00:20:11.080519 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.080496 2575 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-8jwcl" Jul 23 00:20:11.145890 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.145818 2575 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 23 00:20:11.160433 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160399 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-system-cni-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.160567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160444 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.160567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160476 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m845m\" (UniqueName: \"kubernetes.io/projected/562817ed-d6e5-427e-9acf-d9e3804bd86c-kube-api-access-m845m\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.160567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.160567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160531 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-systemd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160556 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-netd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160580 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-config\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160604 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-k8s-cni-cncf-io\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160629 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-bin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160655 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-78z4p\" (UniqueName: \"kubernetes.io/projected/912ac498-7d64-46a8-bc09-5cb60db5ea55-kube-api-access-78z4p\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160717 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-var-lib-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160741 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-bin\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160767 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.160814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160792 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-sys\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160832 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-tuned\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160857 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-tmp\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97511f57-175e-4342-8d95-5abd34d00543-host-slash\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.160955 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vq7zb\" (UniqueName: \"kubernetes.io/projected/97511f57-175e-4342-8d95-5abd34d00543-kube-api-access-vq7zb\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161029 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-system-cni-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161043 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-tuning-conf-dir\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-daemon-config\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161170 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-bin\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161176 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjdqt\" (UniqueName: \"kubernetes.io/projected/9ba635bf-a492-4592-9d61-1666f54a3d7f-kube-api-access-kjdqt\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161195 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-var-lib-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161175 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161208 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/97511f57-175e-4342-8d95-5abd34d00543-host-slash\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-agent-certs\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.161325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161285 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-etc-selinux\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161286 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161287 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-systemd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161317 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac51900d-e12b-494e-a4e9-5e30ebff971d-hosts-file\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161373 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-kubelet-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161381 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161427 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-var-lib-kubelet\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161468 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22prn\" (UniqueName: \"kubernetes.io/projected/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-kube-api-access-22prn\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161485 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-cni-netd\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161526 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-systemd-units\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161627 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-netns\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161630 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-etc-selinux\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161674 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-systemd-units\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161727 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-config\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161727 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-kubelet-config\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161741 2575 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161730 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-netns\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.162237 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161771 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdhmz\" (UniqueName: \"kubernetes.io/projected/e45f1316-4460-4bfe-ad4c-5d78485265da-kube-api-access-sdhmz\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161790 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-kubelet-config\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161858 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-cni-binary-copy\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161885 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-etc-kubernetes\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-cnibin\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161950 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-kubelet\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.161985 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-cnibin\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-os-release\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162018 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-kubelet\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162071 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-kubelet\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162111 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-hostroot\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162144 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7zmpm\" (UniqueName: \"kubernetes.io/projected/a6650f62-d7b7-41ef-b541-b642dac6d011-kube-api-access-7zmpm\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162173 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-dbus\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162203 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-systemd\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162233 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162267 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-slash\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162322 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-socket-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162348 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-system-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162352 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-run-ovn-kubernetes\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162384 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-host-slash\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162418 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-cnibin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162452 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162476 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-serviceca\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-socket-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162546 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-ovn\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162573 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysconfig\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162619 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-run-ovn\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162650 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-conf\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162696 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-run\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162722 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-netns\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162764 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovn-node-metrics-cert\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162791 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-script-lib\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.163978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162816 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162842 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/97511f57-175e-4342-8d95-5abd34d00543-iptables-alerter-script\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162898 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-etc-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162927 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-serviceca\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.162980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-konnectivity-ca\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163004 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163009 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-registration-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163052 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-registration-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163058 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-kubernetes\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163090 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-etc-openvswitch\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163092 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/22e1b738-7df0-4e85-99a9-335b1abd8c66-dbus\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163091 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-node-log\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-node-log\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163154 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-env-overrides\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.163194 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163272 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt4c5\" (UniqueName: \"kubernetes.io/projected/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-kube-api-access-tt4c5\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-device-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163363 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-modprobe-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.164822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163389 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-lib-modules\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163414 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-multus\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163441 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-multus-certs\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163494 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-binary-copy\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163499 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/97511f57-175e-4342-8d95-5abd34d00543-iptables-alerter-script\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163524 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7wg8\" (UniqueName: \"kubernetes.io/projected/ac51900d-e12b-494e-a4e9-5e30ebff971d-kube-api-access-m7wg8\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163548 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-env-overrides\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.163597 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:11.663555545 +0000 UTC m=+2.969703566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163607 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-device-dir\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163708 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-konnectivity-ca\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163549 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-host\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163795 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-conf-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163826 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163878 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-os-release\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163923 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-host\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-log-socket\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.165729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163965 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/912ac498-7d64-46a8-bc09-5cb60db5ea55-os-release\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.163997 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-sys-fs\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164022 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-host\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac51900d-e12b-494e-a4e9-5e30ebff971d-tmp-dir\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-log-socket\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164116 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/a6650f62-d7b7-41ef-b541-b642dac6d011-sys-fs\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164132 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-socket-dir-parent\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/912ac498-7d64-46a8-bc09-5cb60db5ea55-cni-binary-copy\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.166450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.164552 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovnkube-script-lib\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.167345 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.167321 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-ovn-node-metrics-cert\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.167745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.167723 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef-agent-certs\") pod \"konnectivity-agent-ztl74\" (UID: \"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef\") " pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.173240 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.173201 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt4c5\" (UniqueName: \"kubernetes.io/projected/9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c-kube-api-access-tt4c5\") pod \"ovnkube-node-fhw25\" (UID: \"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c\") " pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.173686 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.173505 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22prn\" (UniqueName: \"kubernetes.io/projected/e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8-kube-api-access-22prn\") pod \"node-ca-74sfp\" (UID: \"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8\") " pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.173850 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.173823 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7zmpm\" (UniqueName: \"kubernetes.io/projected/a6650f62-d7b7-41ef-b541-b642dac6d011-kube-api-access-7zmpm\") pod \"aws-ebs-csi-driver-node-8vnr9\" (UID: \"a6650f62-d7b7-41ef-b541-b642dac6d011\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.174282 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.174241 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vq7zb\" (UniqueName: \"kubernetes.io/projected/97511f57-175e-4342-8d95-5abd34d00543-kube-api-access-vq7zb\") pod \"iptables-alerter-cxfpx\" (UID: \"97511f57-175e-4342-8d95-5abd34d00543\") " pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.175100 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.175074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-78z4p\" (UniqueName: \"kubernetes.io/projected/912ac498-7d64-46a8-bc09-5cb60db5ea55-kube-api-access-78z4p\") pod \"multus-additional-cni-plugins-25s8z\" (UID: \"912ac498-7d64-46a8-bc09-5cb60db5ea55\") " pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.241640 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.238840 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"83586bb93cbfd50d0fdc79ded33ba2b5","Type":"ContainerStarted","Data":"857addb30fcfc47db1a5f0e26d027119e16fd23641f681171e77e830eace21d3"} Jul 23 00:20:11.243552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.243482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" event={"ID":"a9ca0bc95fbca063e66eb3b93e200e1c","Type":"ContainerStarted","Data":"daa4dfbce6fd860bdcf4a307f4b99865a52970083a3840d9ade1b4aa86d69d6d"} Jul 23 00:20:11.265344 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265312 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-cnibin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265384 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysconfig\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265409 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-conf\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265432 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-run\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265436 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265454 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-netns\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265500 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265496 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-kubernetes\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265509 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysconfig\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265527 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-modprobe-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265551 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-lib-modules\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265561 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-cnibin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-multus\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265601 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-netns\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265605 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-multus-certs\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265631 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7wg8\" (UniqueName: \"kubernetes.io/projected/ac51900d-e12b-494e-a4e9-5e30ebff971d-kube-api-access-m7wg8\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265650 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-host\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265679 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-conf-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265715 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-conf\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265732 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac51900d-e12b-494e-a4e9-5e30ebff971d-tmp-dir\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265757 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-socket-dir-parent\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265761 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-run\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265789 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m845m\" (UniqueName: \"kubernetes.io/projected/562817ed-d6e5-427e-9acf-d9e3804bd86c-kube-api-access-m845m\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265807 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-multus-certs\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.265893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265819 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-k8s-cni-cncf-io\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265845 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-bin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265871 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-sys\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265855 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-kubernetes\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265908 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-tuned\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265929 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-tmp\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265956 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-daemon-config\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265971 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-modprobe-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.265981 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjdqt\" (UniqueName: \"kubernetes.io/projected/9ba635bf-a492-4592-9d61-1666f54a3d7f-kube-api-access-kjdqt\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266005 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266031 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac51900d-e12b-494e-a4e9-5e30ebff971d-hosts-file\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266045 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-lib-modules\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266055 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-var-lib-kubelet\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266085 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdhmz\" (UniqueName: \"kubernetes.io/projected/e45f1316-4460-4bfe-ad4c-5d78485265da-kube-api-access-sdhmz\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266111 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-cni-binary-copy\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266135 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-etc-kubernetes\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266160 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-os-release\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266186 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-kubelet\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.266792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266216 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-hostroot\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-systemd\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266268 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266294 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-system-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266354 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-host\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266404 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-conf-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266406 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-system-cni-dir\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266877 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac51900d-e12b-494e-a4e9-5e30ebff971d-tmp-dir\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.267027 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-socket-dir-parent\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.267280 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-run-k8s-cni-cncf-io\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.267336 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-sys\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.267732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.267378 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-bin\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268307 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268049 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-multus-daemon-config\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268307 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268246 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-kubelet\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268439 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268415 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-sysctl-d\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.268497 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268446 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-etc-kubernetes\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268497 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ac51900d-e12b-494e-a4e9-5e30ebff971d-hosts-file\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.268601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.266085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-host-var-lib-cni-multus\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-systemd\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.268704 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268617 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-hostroot\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268704 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268685 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/9ba635bf-a492-4592-9d61-1666f54a3d7f-os-release\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268802 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268705 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e45f1316-4460-4bfe-ad4c-5d78485265da-var-lib-kubelet\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.268802 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.268773 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/9ba635bf-a492-4592-9d61-1666f54a3d7f-cni-binary-copy\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.268934 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.268796 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:11.268934 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.268908 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:11.768865923 +0000 UTC m=+3.075013903 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:11.270628 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.270600 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-etc-tuned\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.271473 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.271429 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e45f1316-4460-4bfe-ad4c-5d78485265da-tmp\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.272272 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.272218 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:11.272272 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.272242 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:11.272272 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.272258 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:11.272492 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.272320 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:11.772303536 +0000 UTC m=+3.078451525 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:11.275010 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.274981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7wg8\" (UniqueName: \"kubernetes.io/projected/ac51900d-e12b-494e-a4e9-5e30ebff971d-kube-api-access-m7wg8\") pod \"node-resolver-ldjk4\" (UID: \"ac51900d-e12b-494e-a4e9-5e30ebff971d\") " pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.276668 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.276642 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m845m\" (UniqueName: \"kubernetes.io/projected/562817ed-d6e5-427e-9acf-d9e3804bd86c-kube-api-access-m845m\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.278008 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.277888 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjdqt\" (UniqueName: \"kubernetes.io/projected/9ba635bf-a492-4592-9d61-1666f54a3d7f-kube-api-access-kjdqt\") pod \"multus-q2rrf\" (UID: \"9ba635bf-a492-4592-9d61-1666f54a3d7f\") " pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.279192 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.279149 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdhmz\" (UniqueName: \"kubernetes.io/projected/e45f1316-4460-4bfe-ad4c-5d78485265da-kube-api-access-sdhmz\") pod \"tuned-2pxw5\" (UID: \"e45f1316-4460-4bfe-ad4c-5d78485265da\") " pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.349113 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.349074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" Jul 23 00:20:11.357214 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.357180 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-25s8z" Jul 23 00:20:11.361741 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.361685 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6650f62_d7b7_41ef_b541_b642dac6d011.slice/crio-6b28ae818b5b736f9107ef29b3a3e52b2974a0df6b12a837cd8239b0da6f1a33 WatchSource:0}: Error finding container 6b28ae818b5b736f9107ef29b3a3e52b2974a0df6b12a837cd8239b0da6f1a33: Status 404 returned error can't find the container with id 6b28ae818b5b736f9107ef29b3a3e52b2974a0df6b12a837cd8239b0da6f1a33 Jul 23 00:20:11.368225 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.367717 2575 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 23 00:20:11.368225 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.367903 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-cxfpx" Jul 23 00:20:11.372408 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.372381 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod912ac498_7d64_46a8_bc09_5cb60db5ea55.slice/crio-7955b4a36231bb3e92dcaed3efd17bd6f7040a4db3a75f5f7385452fa04a6373 WatchSource:0}: Error finding container 7955b4a36231bb3e92dcaed3efd17bd6f7040a4db3a75f5f7385452fa04a6373: Status 404 returned error can't find the container with id 7955b4a36231bb3e92dcaed3efd17bd6f7040a4db3a75f5f7385452fa04a6373 Jul 23 00:20:11.375899 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.375881 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:11.378960 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.378920 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97511f57_175e_4342_8d95_5abd34d00543.slice/crio-311574a227944c365dc087c42a4e0dc800f1128874aa6093eb6887388383d295 WatchSource:0}: Error finding container 311574a227944c365dc087c42a4e0dc800f1128874aa6093eb6887388383d295: Status 404 returned error can't find the container with id 311574a227944c365dc087c42a4e0dc800f1128874aa6093eb6887388383d295 Jul 23 00:20:11.382602 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.382573 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:11.391002 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.390980 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-74sfp" Jul 23 00:20:11.395418 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.395393 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e16c15a_e9ce_4d4a_a41a_bc6c1cfb759c.slice/crio-a6514ef48b0eb95c42d1db79ed18996f8f1d37c24d01153031e6cab238cdad4c WatchSource:0}: Error finding container a6514ef48b0eb95c42d1db79ed18996f8f1d37c24d01153031e6cab238cdad4c: Status 404 returned error can't find the container with id a6514ef48b0eb95c42d1db79ed18996f8f1d37c24d01153031e6cab238cdad4c Jul 23 00:20:11.398440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.398235 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-ldjk4" Jul 23 00:20:11.402307 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.402277 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b5bab9e_600d_40e6_a6f2_57cb9fcf0bef.slice/crio-95e75029318581918f4cd337a9e1a47dba2706c48c4eeee93cd42adbf192a707 WatchSource:0}: Error finding container 95e75029318581918f4cd337a9e1a47dba2706c48c4eeee93cd42adbf192a707: Status 404 returned error can't find the container with id 95e75029318581918f4cd337a9e1a47dba2706c48c4eeee93cd42adbf192a707 Jul 23 00:20:11.402893 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.402865 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode44b9dfb_a0a3_4d0d_be49_2a7fb85e8ba8.slice/crio-508e069a392813be291dd12f5c900481d2b66f946449c11a47f0cec0cb1540a2 WatchSource:0}: Error finding container 508e069a392813be291dd12f5c900481d2b66f946449c11a47f0cec0cb1540a2: Status 404 returned error can't find the container with id 508e069a392813be291dd12f5c900481d2b66f946449c11a47f0cec0cb1540a2 Jul 23 00:20:11.406006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.405477 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" Jul 23 00:20:11.410222 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.410191 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac51900d_e12b_494e_a4e9_5e30ebff971d.slice/crio-0a7a02b27b69401873dcf46d0fd623a554cffc2e7153ce217aca98d200529988 WatchSource:0}: Error finding container 0a7a02b27b69401873dcf46d0fd623a554cffc2e7153ce217aca98d200529988: Status 404 returned error can't find the container with id 0a7a02b27b69401873dcf46d0fd623a554cffc2e7153ce217aca98d200529988 Jul 23 00:20:11.411401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.411380 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-q2rrf" Jul 23 00:20:11.417946 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.417878 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode45f1316_4460_4bfe_ad4c_5d78485265da.slice/crio-0160ca18f556179836ebe917862e6ec43733bcb0fc9df7dac8b51c108f58c059 WatchSource:0}: Error finding container 0160ca18f556179836ebe917862e6ec43733bcb0fc9df7dac8b51c108f58c059: Status 404 returned error can't find the container with id 0160ca18f556179836ebe917862e6ec43733bcb0fc9df7dac8b51c108f58c059 Jul 23 00:20:11.424136 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:11.424099 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ba635bf_a492_4592_9d61_1666f54a3d7f.slice/crio-a812f6dddbe8886a64d6b03b6db4b3b69c7f87e42999a8d2f34de453f6616e69 WatchSource:0}: Error finding container a812f6dddbe8886a64d6b03b6db4b3b69c7f87e42999a8d2f34de453f6616e69: Status 404 returned error can't find the container with id a812f6dddbe8886a64d6b03b6db4b3b69c7f87e42999a8d2f34de453f6616e69 Jul 23 00:20:11.669042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.668936 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:11.669205 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.669152 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:11.669280 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.669222 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:12.669201386 +0000 UTC m=+3.975349377 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:11.770004 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.769967 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:11.770205 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.770118 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:11.770205 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.770195 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:12.77017354 +0000 UTC m=+4.076321530 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:11.871205 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:11.871168 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:11.871419 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.871385 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:11.871419 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.871420 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:11.871589 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.871436 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:11.871589 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:11.871505 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:12.871484969 +0000 UTC m=+4.177632945 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:12.083209 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.083115 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 00:15:11 +0000 UTC" deadline="2028-03-02 05:10:45.761406944 +0000 UTC" Jul 23 00:20:12.083209 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.083149 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14116h50m33.678261531s" Jul 23 00:20:12.231367 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.231322 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:12.231548 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.231478 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:12.248179 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.248092 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" event={"ID":"e45f1316-4460-4bfe-ad4c-5d78485265da","Type":"ContainerStarted","Data":"0160ca18f556179836ebe917862e6ec43733bcb0fc9df7dac8b51c108f58c059"} Jul 23 00:20:12.249921 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.249889 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerStarted","Data":"7955b4a36231bb3e92dcaed3efd17bd6f7040a4db3a75f5f7385452fa04a6373"} Jul 23 00:20:12.251284 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.251259 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" event={"ID":"a6650f62-d7b7-41ef-b541-b642dac6d011","Type":"ContainerStarted","Data":"6b28ae818b5b736f9107ef29b3a3e52b2974a0df6b12a837cd8239b0da6f1a33"} Jul 23 00:20:12.252752 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.252725 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2rrf" event={"ID":"9ba635bf-a492-4592-9d61-1666f54a3d7f","Type":"ContainerStarted","Data":"a812f6dddbe8886a64d6b03b6db4b3b69c7f87e42999a8d2f34de453f6616e69"} Jul 23 00:20:12.254568 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.254538 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ldjk4" event={"ID":"ac51900d-e12b-494e-a4e9-5e30ebff971d","Type":"ContainerStarted","Data":"0a7a02b27b69401873dcf46d0fd623a554cffc2e7153ce217aca98d200529988"} Jul 23 00:20:12.255935 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.255912 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-74sfp" event={"ID":"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8","Type":"ContainerStarted","Data":"508e069a392813be291dd12f5c900481d2b66f946449c11a47f0cec0cb1540a2"} Jul 23 00:20:12.257492 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.257468 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ztl74" event={"ID":"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef","Type":"ContainerStarted","Data":"95e75029318581918f4cd337a9e1a47dba2706c48c4eeee93cd42adbf192a707"} Jul 23 00:20:12.258968 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.258921 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"a6514ef48b0eb95c42d1db79ed18996f8f1d37c24d01153031e6cab238cdad4c"} Jul 23 00:20:12.260535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.260508 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxfpx" event={"ID":"97511f57-175e-4342-8d95-5abd34d00543","Type":"ContainerStarted","Data":"311574a227944c365dc087c42a4e0dc800f1128874aa6093eb6887388383d295"} Jul 23 00:20:12.679580 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.678948 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:12.679580 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.679125 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:12.679580 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.679192 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:14.679172178 +0000 UTC m=+5.985320158 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:12.780161 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.780120 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:12.780430 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.780276 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:12.780430 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.780345 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:14.780324594 +0000 UTC m=+6.086472574 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:12.882087 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:12.881401 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:12.882087 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.881608 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:12.882087 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.881630 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:12.882087 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.881646 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:12.882087 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:12.881706 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:14.881685708 +0000 UTC m=+6.187833686 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:13.084563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.084334 2575 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-22 00:15:11 +0000 UTC" deadline="2028-05-06 12:14:18.540710418 +0000 UTC" Jul 23 00:20:13.084563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.084554 2575 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15683h54m5.456161843s" Jul 23 00:20:13.230793 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.230658 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:13.230958 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:13.230807 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:13.231270 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.231112 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:13.231270 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:13.231223 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:13.285201 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.284393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" event={"ID":"a9ca0bc95fbca063e66eb3b93e200e1c","Type":"ContainerStarted","Data":"b03e53705dcbb632f387bfaea748147595d84c01ed6711d6178d57f64080e0b2"} Jul 23 00:20:13.299122 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:13.299070 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-87.ec2.internal" podStartSLOduration=2.299055011 podStartE2EDuration="2.299055011s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:13.298690503 +0000 UTC m=+4.604838503" watchObservedRunningTime="2026-07-23 00:20:13.299055011 +0000 UTC m=+4.605203008" Jul 23 00:20:14.230745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.230712 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:14.231277 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.230875 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:14.298662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.298625 2575 generic.go:356] "Generic (PLEG): container finished" podID="83586bb93cbfd50d0fdc79ded33ba2b5" containerID="41e663e1a4d2ab04d59d68c858944c7a393cdd46ae20376221757702cc7b2c32" exitCode=0 Jul 23 00:20:14.298828 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.298714 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"83586bb93cbfd50d0fdc79ded33ba2b5","Type":"ContainerDied","Data":"41e663e1a4d2ab04d59d68c858944c7a393cdd46ae20376221757702cc7b2c32"} Jul 23 00:20:14.703887 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.703805 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:14.704078 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.703987 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:14.704078 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.704052 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:18.704030493 +0000 UTC m=+10.010178492 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:14.804414 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.804378 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:14.804603 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.804518 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:14.804603 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.804579 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:18.804559604 +0000 UTC m=+10.110707579 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:14.905350 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:14.905314 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:14.905544 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.905505 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:14.905544 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.905535 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:14.905669 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.905548 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:14.905669 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:14.905617 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:18.905595619 +0000 UTC m=+10.211743613 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:15.232041 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:15.232011 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:15.232532 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:15.232164 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:15.232532 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:15.232251 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:15.232532 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:15.232375 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:16.231614 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:16.231081 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:16.231614 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:16.231231 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:17.231274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:17.231233 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:17.231758 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:17.231289 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:17.231758 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:17.231401 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:17.231758 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:17.231598 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:18.230879 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:18.230652 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:18.230879 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.230795 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:18.741531 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:18.741489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:18.742059 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.741648 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:18.742059 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.741723 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:26.741703274 +0000 UTC m=+18.047851263 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:18.842321 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:18.842236 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:18.842483 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.842415 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:18.842549 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.842498 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:26.842475128 +0000 UTC m=+18.148623116 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:18.942844 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:18.942747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:18.943043 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.942959 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:18.943043 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.942990 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:18.943043 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.943003 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:18.943213 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:18.943065 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:26.943045997 +0000 UTC m=+18.249193974 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:19.231525 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:19.231435 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:19.231706 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:19.231558 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:19.231706 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:19.231626 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:19.231706 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:19.231644 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:20.231213 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:20.231170 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:20.231688 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:20.231307 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:21.231055 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:21.231008 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:21.231255 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:21.231165 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:21.231632 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:21.231590 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:21.231746 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:21.231717 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:22.231135 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:22.230911 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:22.231135 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:22.231049 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:23.230647 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:23.230618 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:23.230788 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:23.230739 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:23.230871 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:23.230812 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:23.230926 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:23.230914 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:24.231182 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:24.231152 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:24.231576 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:24.231255 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:25.230735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:25.230699 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:25.230963 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:25.230709 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:25.230963 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:25.230857 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:25.230963 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:25.230942 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:26.230618 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:26.230588 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:26.231181 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:26.230709 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:26.802403 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:26.802370 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:26.802574 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:26.802516 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:26.802627 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:26.802582 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:42.802565574 +0000 UTC m=+34.108713550 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:26.903294 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:26.903262 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:26.903449 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:26.903376 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:26.903449 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:26.903443 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:20:42.903423668 +0000 UTC m=+34.209571664 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:27.004239 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:27.004198 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:27.004411 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.004367 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:27.004411 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.004386 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:27.004411 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.004399 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:27.004568 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.004461 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:20:43.004443966 +0000 UTC m=+34.310591956 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:27.231311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:27.231228 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:27.231311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:27.231242 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:27.231762 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.231360 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:27.231762 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:27.231515 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:28.231054 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:28.231021 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:28.231244 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:28.231155 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:29.231541 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:29.231509 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:29.232045 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:29.231618 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:29.232045 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:29.231716 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:29.232045 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:29.231803 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:30.230936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:30.230905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:30.231076 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:30.231007 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:31.230595 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:31.230568 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:31.230929 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:31.230704 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:31.230929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:31.230740 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:31.230929 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:31.230810 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:32.230880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.230493 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:32.231701 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:32.230929 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:32.342470 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.342439 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="a1ea0ba20b41fe21ec6997fe77b596e111db50b1e27bcf28ea916d55c8580d0a" exitCode=0 Jul 23 00:20:32.342613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.342506 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"a1ea0ba20b41fe21ec6997fe77b596e111db50b1e27bcf28ea916d55c8580d0a"} Jul 23 00:20:32.344048 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.344026 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" event={"ID":"a6650f62-d7b7-41ef-b541-b642dac6d011","Type":"ContainerStarted","Data":"92116efa14f06f5b82cc99814e1e35f45a0c9eaeb829448850d60e2dabb3b2ec"} Jul 23 00:20:32.345374 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.345352 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-q2rrf" event={"ID":"9ba635bf-a492-4592-9d61-1666f54a3d7f","Type":"ContainerStarted","Data":"0e5703f9467c3fcfe5f79f8ff7bdf1149f3e70b62f4d41577256f3042c64e2ab"} Jul 23 00:20:32.346789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.346770 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-ldjk4" event={"ID":"ac51900d-e12b-494e-a4e9-5e30ebff971d","Type":"ContainerStarted","Data":"7599c9a7c0adc059d6ab070c46aa931b52a086e9cf4e139a206342c86ec60ae5"} Jul 23 00:20:32.348152 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.348109 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-74sfp" event={"ID":"e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8","Type":"ContainerStarted","Data":"319e8c4b93c8baf216968ed9678b81c4c8e6d25a98ed099552b937e2d81d28ee"} Jul 23 00:20:32.349358 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.349334 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ztl74" event={"ID":"9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef","Type":"ContainerStarted","Data":"2af5b55f1a1c888d8987e02d95d64f4715a3b1bf9d917b071e3dc8485fe7598f"} Jul 23 00:20:32.352093 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352078 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:20:32.352408 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352390 2575 generic.go:356] "Generic (PLEG): container finished" podID="9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c" containerID="90e8511d1e6efd5e26c92af2d2ecef0694f1a4dcd2ba4b69ee308fec8e2b7970" exitCode=1 Jul 23 00:20:32.352482 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352457 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"e0996631621c90c593e97c2cfc1c311932a3d1bd7a95374fd1be5c07326be309"} Jul 23 00:20:32.352581 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352517 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"bcc75b80878d49289dde5a763cb617051caaff07b2a1534ea8ced2fa7b8feeb6"} Jul 23 00:20:32.352581 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352568 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"5359712b02e86a96c2afbb9653ae464d4063a69a89ac0146a2177f976457e45a"} Jul 23 00:20:32.352680 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352599 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"548d2a97bcab57af8166818721d76759b57acbef03b62f3dce90ec5bf03d1083"} Jul 23 00:20:32.352680 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352631 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerDied","Data":"90e8511d1e6efd5e26c92af2d2ecef0694f1a4dcd2ba4b69ee308fec8e2b7970"} Jul 23 00:20:32.352680 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.352670 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"c34146796ce6243cd1ba91953e38b0f2f11c14a43ba0a69f8ab6063ce4470fd9"} Jul 23 00:20:32.354028 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.354002 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" event={"ID":"83586bb93cbfd50d0fdc79ded33ba2b5","Type":"ContainerStarted","Data":"21f14a1aac065160baa0fee53b842d3a8da9928298670e6947499d3274dc7797"} Jul 23 00:20:32.355433 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.355415 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" event={"ID":"e45f1316-4460-4bfe-ad4c-5d78485265da","Type":"ContainerStarted","Data":"0dfbe07413b944b2187ff06289b4fdf5b531d94d3dacf7ca222fc1ec1d1a91c2"} Jul 23 00:20:32.382696 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.382648 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-ldjk4" podStartSLOduration=2.402060853 podStartE2EDuration="22.382632442s" totalImagesPullingTime="19.980571589s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.38232947 +0000 UTC m=+23.688477467" watchObservedRunningTime="2026-07-23 00:20:32.382632442 +0000 UTC m=+23.688780437" Jul 23 00:20:32.399761 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.399716 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-q2rrf" podStartSLOduration=1.951205457 podStartE2EDuration="22.399703949s" totalImagesPullingTime="20.448498492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.399625961 +0000 UTC m=+23.705773973" watchObservedRunningTime="2026-07-23 00:20:32.399703949 +0000 UTC m=+23.705851946" Jul 23 00:20:32.415510 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.415464 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-87.ec2.internal" podStartSLOduration=21.415450548 podStartE2EDuration="21.415450548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.415011471 +0000 UTC m=+23.721159480" watchObservedRunningTime="2026-07-23 00:20:32.415450548 +0000 UTC m=+23.721598542" Jul 23 00:20:32.430299 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.430256 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ztl74" podStartSLOduration=6.9809556409999995 podStartE2EDuration="22.430244718s" totalImagesPullingTime="15.449289077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.430008596 +0000 UTC m=+23.736156590" watchObservedRunningTime="2026-07-23 00:20:32.430244718 +0000 UTC m=+23.736392711" Jul 23 00:20:32.446211 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.446172 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-74sfp" podStartSLOduration=2.456412224 podStartE2EDuration="22.44615666s" totalImagesPullingTime="19.989744436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.445512781 +0000 UTC m=+23.751660794" watchObservedRunningTime="2026-07-23 00:20:32.44615666 +0000 UTC m=+23.752304650" Jul 23 00:20:32.463719 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:32.463678 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-2pxw5" podStartSLOduration=2.471897431 podStartE2EDuration="22.463661426s" totalImagesPullingTime="19.991763995s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:32.463251283 +0000 UTC m=+23.769399278" watchObservedRunningTime="2026-07-23 00:20:32.463661426 +0000 UTC m=+23.769809424" Jul 23 00:20:33.074404 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.074378 2575 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 23 00:20:33.115267 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.115168 2575 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-23T00:20:33.074397818Z","UUID":"3b341858-4993-4166-b028-cc686ad96d51","Handler":null,"Name":"","Endpoint":""} Jul 23 00:20:33.116587 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.116569 2575 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 23 00:20:33.116678 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.116606 2575 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 23 00:20:33.230622 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.230562 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:33.230749 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.230566 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:33.230749 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:33.230666 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:33.230885 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:33.230747 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:33.358982 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.358953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-cxfpx" event={"ID":"97511f57-175e-4342-8d95-5abd34d00543","Type":"ContainerStarted","Data":"f9f3bf11686d1cd049625476c86c53186ee114291c78616d6dec76a719263015"} Jul 23 00:20:33.361207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.361166 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" event={"ID":"a6650f62-d7b7-41ef-b541-b642dac6d011","Type":"ContainerStarted","Data":"b05fbe2c5a2f33fb0ea79e5bc6dc179664131ef6b9e465fdf173df4f4774281d"} Jul 23 00:20:33.374465 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:33.374420 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-cxfpx" podStartSLOduration=3.360762953 podStartE2EDuration="23.374402434s" totalImagesPullingTime="20.013639481s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:33.374123333 +0000 UTC m=+24.680271327" watchObservedRunningTime="2026-07-23 00:20:33.374402434 +0000 UTC m=+24.680550431" Jul 23 00:20:34.230621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:34.230585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:34.230785 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:34.230719 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:34.366543 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:34.366356 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:20:34.367002 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:34.366964 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"1fb86339bf5e8f53722ef15f8bdca821f8fe8da949a95a6cd37acddf7f5c5261"} Jul 23 00:20:35.230907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.230869 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:35.231073 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.230909 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:35.231073 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:35.231018 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:35.231165 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:35.231122 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:35.371654 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.371619 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" event={"ID":"a6650f62-d7b7-41ef-b541-b642dac6d011","Type":"ContainerStarted","Data":"19a169b4959f47c6b83932b6537221bb21b805415ca50d484e423abc20436451"} Jul 23 00:20:35.390943 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.390887 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-8vnr9" podStartSLOduration=2.719252833 podStartE2EDuration="25.390867193s" totalImagesPullingTime="22.67161436s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:35.389855252 +0000 UTC m=+26.696003246" watchObservedRunningTime="2026-07-23 00:20:35.390867193 +0000 UTC m=+26.697015188" Jul 23 00:20:35.696171 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.696083 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:35.696725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:35.696702 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:36.231111 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:36.231086 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:36.231222 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:36.231198 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:36.374000 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:36.373972 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:36.374585 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:36.374273 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ztl74" Jul 23 00:20:37.230560 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:37.230539 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:37.230651 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:37.230539 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:37.230651 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:37.230638 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:37.230730 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:37.230710 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:37.379012 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:37.378811 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:20:37.379498 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:37.379375 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"f35801e37fedb80fb6565afec7dceaa0887b2b928bd6d8e72c1172b90f66d308"} Jul 23 00:20:37.379848 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:37.379814 2575 scope.go:117] "RemoveContainer" containerID="90e8511d1e6efd5e26c92af2d2ecef0694f1a4dcd2ba4b69ee308fec8e2b7970" Jul 23 00:20:38.230624 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.230599 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:38.230782 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:38.230687 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:38.385094 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.385032 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:20:38.385502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.385459 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" event={"ID":"9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c","Type":"ContainerStarted","Data":"9a639e12e7fecafae65ade10c0ae6a97c4257428ebc46099e8af766f8e8dbdd3"} Jul 23 00:20:38.385564 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.385538 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:38.385716 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.385700 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:38.385818 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.385773 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:38.387476 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.387449 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="7ac21b3314f0cffc78d3e6d3588823481321bc083537940d82301f1337bc21b1" exitCode=0 Jul 23 00:20:38.387579 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.387522 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"7ac21b3314f0cffc78d3e6d3588823481321bc083537940d82301f1337bc21b1"} Jul 23 00:20:38.400922 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.400903 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:38.401001 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.400991 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:20:38.412728 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:38.412689 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" podStartSLOduration=8.338379856 podStartE2EDuration="28.412676464s" totalImagesPullingTime="20.074296608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:38.411283273 +0000 UTC m=+29.717431267" watchObservedRunningTime="2026-07-23 00:20:38.412676464 +0000 UTC m=+29.718824459" Jul 23 00:20:39.233041 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:39.233009 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:39.233206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:39.233013 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:39.233206 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:39.233111 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:39.233206 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:39.233167 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:40.230792 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:40.230761 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:40.231158 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:40.230876 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:40.394092 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:40.394059 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="1fc6d3abadcd94fafbb511cfd0532985d8d42c7f2fc58222fe0c240f0ee69fa0" exitCode=0 Jul 23 00:20:40.394239 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:40.394147 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"1fc6d3abadcd94fafbb511cfd0532985d8d42c7f2fc58222fe0c240f0ee69fa0"} Jul 23 00:20:41.233135 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:41.233110 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:41.233467 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:41.233141 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:41.233467 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:41.233193 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:41.233467 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:41.233340 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:42.230818 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:42.230797 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:42.230976 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:42.230908 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:42.400485 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:42.400457 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="738060cfb867e25266921a9fa071f38b6dc8abdae2d6cde3786d24f08fbcd9e0" exitCode=0 Jul 23 00:20:42.400865 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:42.400494 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"738060cfb867e25266921a9fa071f38b6dc8abdae2d6cde3786d24f08fbcd9e0"} Jul 23 00:20:42.822839 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:42.822805 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:42.823009 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:42.822971 2575 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:42.823093 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:42.823050 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret podName:22e1b738-7df0-4e85-99a9-335b1abd8c66 nodeName:}" failed. No retries permitted until 2026-07-23 00:21:14.823029122 +0000 UTC m=+66.129177097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret") pod "global-pull-secret-syncer-htfz9" (UID: "22e1b738-7df0-4e85-99a9-335b1abd8c66") : object "kube-system"/"original-pull-secret" not registered Jul 23 00:20:42.924096 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:42.924060 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:42.924231 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:42.924175 2575 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:42.924288 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:42.924241 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs podName:562817ed-d6e5-427e-9acf-d9e3804bd86c nodeName:}" failed. No retries permitted until 2026-07-23 00:21:14.924222943 +0000 UTC m=+66.230370916 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs") pod "network-metrics-daemon-krcjb" (UID: "562817ed-d6e5-427e-9acf-d9e3804bd86c") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 23 00:20:43.024776 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:43.024747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:43.024939 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.024889 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 23 00:20:43.024939 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.024905 2575 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 23 00:20:43.024939 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.024917 2575 projected.go:196] Error preparing data for projected volume kube-api-access-5tl2m for pod openshift-network-diagnostics/network-check-target-945mz: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:43.025103 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.024977 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m podName:30602cc0-76d5-4e02-b766-9b522b8c7b96 nodeName:}" failed. No retries permitted until 2026-07-23 00:21:15.024958897 +0000 UTC m=+66.331106872 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-5tl2m" (UniqueName: "kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m") pod "network-check-target-945mz" (UID: "30602cc0-76d5-4e02-b766-9b522b8c7b96") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 23 00:20:43.230677 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:43.230614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:43.230797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:43.230614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:43.230864 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.230709 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:43.230909 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:43.230854 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:44.230822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:44.230787 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:44.231299 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:44.230926 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:45.231040 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:45.231005 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:45.231493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:45.231014 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:45.231493 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:45.231131 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:45.231493 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:45.231258 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:46.230986 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:46.230953 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:46.231164 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:46.231063 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:47.230551 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:47.230526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:47.230673 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:47.230526 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:47.230732 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:47.230664 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:47.230732 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:47.230691 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:47.416268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:47.416194 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="45c2bdb60a8da28aa590a53e95180facb562d2d2c32f57be965af7737d8f20df" exitCode=0 Jul 23 00:20:47.416268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:47.416241 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"45c2bdb60a8da28aa590a53e95180facb562d2d2c32f57be965af7737d8f20df"} Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.203459 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-945mz"] Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.203573 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-htfz9"] Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.203703 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:48.203879 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.204093 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:48.204718 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:48.204219 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:48.206071 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.206040 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-krcjb"] Jul 23 00:20:48.206195 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.206160 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:48.206301 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:48.206259 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:48.421249 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.421171 2575 generic.go:356] "Generic (PLEG): container finished" podID="912ac498-7d64-46a8-bc09-5cb60db5ea55" containerID="d149bfa71dd458ab67d75d952f3da493503ad97f124c0d79f64fb7e0c0fd65cf" exitCode=0 Jul 23 00:20:48.421249 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:48.421223 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerDied","Data":"d149bfa71dd458ab67d75d952f3da493503ad97f124c0d79f64fb7e0c0fd65cf"} Jul 23 00:20:49.426110 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:49.425953 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-25s8z" event={"ID":"912ac498-7d64-46a8-bc09-5cb60db5ea55","Type":"ContainerStarted","Data":"93da72c3bdd9dec9987065cfd6a7d4bc4b9044e13bcae44f4f8e38a0339ad5cc"} Jul 23 00:20:49.447406 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:49.447364 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-25s8z" podStartSLOduration=7.4389972140000005 podStartE2EDuration="39.447351737s" totalImagesPullingTime="32.008354523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:49.446859136 +0000 UTC m=+40.753007129" watchObservedRunningTime="2026-07-23 00:20:49.447351737 +0000 UTC m=+40.753499730" Jul 23 00:20:50.231103 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:50.231075 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:50.231249 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:50.231084 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:50.231249 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:50.231183 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:50.231249 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:50.231173 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-945mz" podUID="30602cc0-76d5-4e02-b766-9b522b8c7b96" Jul 23 00:20:50.231360 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:50.231255 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-krcjb" podUID="562817ed-d6e5-427e-9acf-d9e3804bd86c" Jul 23 00:20:50.231360 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:20:50.231335 2575 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-htfz9" podUID="22e1b738-7df0-4e85-99a9-335b1abd8c66" Jul 23 00:20:51.568781 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.568757 2575 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-87.ec2.internal" event="NodeReady" Jul 23 00:20:51.569133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.568872 2575 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 23 00:20:51.599718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.599695 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5bd67c9c9f-44p27"] Jul 23 00:20:51.628428 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.628408 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-6q89s"] Jul 23 00:20:51.628538 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.628521 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.631079 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.631058 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 23 00:20:51.631196 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.631102 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8wqt5\"" Jul 23 00:20:51.631196 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.631182 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 23 00:20:51.631328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.631238 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 23 00:20:51.636129 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.636066 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 23 00:20:51.642446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.642428 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gmhmg"] Jul 23 00:20:51.642564 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.642549 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.646304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.645214 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-t68f8\"" Jul 23 00:20:51.646304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.645480 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 23 00:20:51.646304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.645726 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 23 00:20:51.646304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.645964 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 23 00:20:51.646304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.646176 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 23 00:20:51.664668 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.664648 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd67c9c9f-44p27"] Jul 23 00:20:51.664751 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.664672 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6q89s"] Jul 23 00:20:51.664751 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.664693 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmhmg"] Jul 23 00:20:51.664858 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.664752 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.667171 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.667153 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 23 00:20:51.667280 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.667185 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 23 00:20:51.667280 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.667191 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rxnz4\"" Jul 23 00:20:51.667280 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.667245 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 23 00:20:51.714146 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.714124 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-g55qm"] Jul 23 00:20:51.724694 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.724671 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.725982 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.725915 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g55qm"] Jul 23 00:20:51.727084 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.727068 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 23 00:20:51.728059 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.728040 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 23 00:20:51.728152 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.728114 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hzp88\"" Jul 23 00:20:51.793002 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.792980 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4tl4\" (UniqueName: \"kubernetes.io/projected/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-kube-api-access-z4tl4\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.793078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793009 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-tls\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7h2zh\" (UniqueName: \"kubernetes.io/projected/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-api-access-7h2zh\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.793078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793056 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-certificates\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793198 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793104 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.793198 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793167 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-bound-sa-token\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793198 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-crio-socket\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.793294 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793238 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-trusted-ca\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793294 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793270 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13b3c005-9f4b-49b2-b193-ac8df029ac6e-ca-trust-extracted\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793360 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793292 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-cert\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.793360 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpbhh\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-kube-api-access-mpbhh\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793364 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-data-volume\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.793425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793380 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.793496 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793429 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-image-registry-private-configuration\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.793496 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.793446 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-installation-pull-secrets\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-894km\" (UniqueName: \"kubernetes.io/projected/36de04af-c095-46e4-ae46-4b049fda43fc-kube-api-access-894km\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.894335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894313 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-cert\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.894335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpbhh\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-kube-api-access-mpbhh\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894363 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-data-volume\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.894502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894386 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.894502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894410 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36de04af-c095-46e4-ae46-4b049fda43fc-config-volume\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.894502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-image-registry-private-configuration\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894480 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-installation-pull-secrets\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894543 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4tl4\" (UniqueName: \"kubernetes.io/projected/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-kube-api-access-z4tl4\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894590 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-tls\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894627 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7h2zh\" (UniqueName: \"kubernetes.io/projected/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-api-access-7h2zh\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894661 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-certificates\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894687 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.894735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894718 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-bound-sa-token\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894742 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-crio-socket\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894779 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-trusted-ca\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894807 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36de04af-c095-46e4-ae46-4b049fda43fc-metrics-tls\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13b3c005-9f4b-49b2-b193-ac8df029ac6e-ca-trust-extracted\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894900 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36de04af-c095-46e4-ae46-4b049fda43fc-tmp-dir\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.895042 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.894959 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-data-volume\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.895391 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.895275 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.895757 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.895550 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-certificates\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.895857 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.895806 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13b3c005-9f4b-49b2-b193-ac8df029ac6e-trusted-ca\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.895939 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.895918 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-crio-socket\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.896139 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.896122 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13b3c005-9f4b-49b2-b193-ac8df029ac6e-ca-trust-extracted\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.898800 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.898770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.898923 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.898896 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-cert\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.898987 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.898936 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-image-registry-private-configuration\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.899031 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.898981 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13b3c005-9f4b-49b2-b193-ac8df029ac6e-installation-pull-secrets\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.899031 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.899020 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-registry-tls\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.902197 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.902177 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-bound-sa-token\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.902505 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.902486 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpbhh\" (UniqueName: \"kubernetes.io/projected/13b3c005-9f4b-49b2-b193-ac8df029ac6e-kube-api-access-mpbhh\") pod \"image-registry-5bd67c9c9f-44p27\" (UID: \"13b3c005-9f4b-49b2-b193-ac8df029ac6e\") " pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.902505 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.902489 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4tl4\" (UniqueName: \"kubernetes.io/projected/5cfcd5ef-d275-4d6a-b7ca-1456b86dd751-kube-api-access-z4tl4\") pod \"ingress-canary-gmhmg\" (UID: \"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751\") " pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.902983 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.902964 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7h2zh\" (UniqueName: \"kubernetes.io/projected/a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd-kube-api-access-7h2zh\") pod \"insights-runtime-extractor-6q89s\" (UID: \"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd\") " pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.949436 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.949416 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:51.952542 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.952521 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-6q89s" Jul 23 00:20:51.972657 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.972638 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gmhmg" Jul 23 00:20:51.995964 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.995937 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36de04af-c095-46e4-ae46-4b049fda43fc-metrics-tls\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.996047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.995990 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36de04af-c095-46e4-ae46-4b049fda43fc-tmp-dir\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.996047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.996022 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-894km\" (UniqueName: \"kubernetes.io/projected/36de04af-c095-46e4-ae46-4b049fda43fc-kube-api-access-894km\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.996110 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.996067 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36de04af-c095-46e4-ae46-4b049fda43fc-config-volume\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.996288 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.996272 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/36de04af-c095-46e4-ae46-4b049fda43fc-tmp-dir\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.996675 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.996653 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/36de04af-c095-46e4-ae46-4b049fda43fc-config-volume\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:51.998105 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:51.998085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/36de04af-c095-46e4-ae46-4b049fda43fc-metrics-tls\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:52.003819 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.003803 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-894km\" (UniqueName: \"kubernetes.io/projected/36de04af-c095-46e4-ae46-4b049fda43fc-kube-api-access-894km\") pod \"dns-default-g55qm\" (UID: \"36de04af-c095-46e4-ae46-4b049fda43fc\") " pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:52.033960 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.033934 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:52.163166 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.163013 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5bd67c9c9f-44p27"] Jul 23 00:20:52.167360 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:52.167327 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13b3c005_9f4b_49b2_b193_ac8df029ac6e.slice/crio-ece3b27912e13a24e3051529764178f98dee9b73385285d2437d9a4675078b0b WatchSource:0}: Error finding container ece3b27912e13a24e3051529764178f98dee9b73385285d2437d9a4675078b0b: Status 404 returned error can't find the container with id ece3b27912e13a24e3051529764178f98dee9b73385285d2437d9a4675078b0b Jul 23 00:20:52.176076 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.176031 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-6q89s"] Jul 23 00:20:52.178700 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:52.178671 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3a0ceb9_8c14_4d1e_bf13_4319d7b6f4bd.slice/crio-3076b4fdb7aea383e5781714e8b27fcd7a176fae0dd26916f19ece04a12377d1 WatchSource:0}: Error finding container 3076b4fdb7aea383e5781714e8b27fcd7a176fae0dd26916f19ece04a12377d1: Status 404 returned error can't find the container with id 3076b4fdb7aea383e5781714e8b27fcd7a176fae0dd26916f19ece04a12377d1 Jul 23 00:20:52.198187 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.198162 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gmhmg"] Jul 23 00:20:52.202131 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:52.202108 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5cfcd5ef_d275_4d6a_b7ca_1456b86dd751.slice/crio-0e15db3abce3b5014c7a156c7257fe47960988d75c5e4c61be4f5d76a7d8a199 WatchSource:0}: Error finding container 0e15db3abce3b5014c7a156c7257fe47960988d75c5e4c61be4f5d76a7d8a199: Status 404 returned error can't find the container with id 0e15db3abce3b5014c7a156c7257fe47960988d75c5e4c61be4f5d76a7d8a199 Jul 23 00:20:52.215693 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.215637 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-g55qm"] Jul 23 00:20:52.219886 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:20:52.219863 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod36de04af_c095_46e4_ae46_4b049fda43fc.slice/crio-f3c3ce050b8443fe7b3619881a0ea67d8758e3661ffd0de379abc826f7aff35d WatchSource:0}: Error finding container f3c3ce050b8443fe7b3619881a0ea67d8758e3661ffd0de379abc826f7aff35d: Status 404 returned error can't find the container with id f3c3ce050b8443fe7b3619881a0ea67d8758e3661ffd0de379abc826f7aff35d Jul 23 00:20:52.230434 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.230415 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:20:52.230434 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.230430 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:20:52.230585 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.230452 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233107 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233165 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233197 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233220 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2ns5k\"" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233196 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 00:20:52.233303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.233270 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-52cgs\"" Jul 23 00:20:52.434113 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.434039 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g55qm" event={"ID":"36de04af-c095-46e4-ae46-4b049fda43fc","Type":"ContainerStarted","Data":"f3c3ce050b8443fe7b3619881a0ea67d8758e3661ffd0de379abc826f7aff35d"} Jul 23 00:20:52.435176 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.435148 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6q89s" event={"ID":"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd","Type":"ContainerStarted","Data":"3076b4fdb7aea383e5781714e8b27fcd7a176fae0dd26916f19ece04a12377d1"} Jul 23 00:20:52.436637 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.436611 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" event={"ID":"13b3c005-9f4b-49b2-b193-ac8df029ac6e","Type":"ContainerStarted","Data":"bc277c7f5c6ad0fdac33b010c06721c7ec0e6ce0451c0285a37d0621c09c2a55"} Jul 23 00:20:52.436744 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.436648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" event={"ID":"13b3c005-9f4b-49b2-b193-ac8df029ac6e","Type":"ContainerStarted","Data":"ece3b27912e13a24e3051529764178f98dee9b73385285d2437d9a4675078b0b"} Jul 23 00:20:52.436744 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.436674 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:20:52.437796 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.437761 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmhmg" event={"ID":"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751","Type":"ContainerStarted","Data":"0e15db3abce3b5014c7a156c7257fe47960988d75c5e4c61be4f5d76a7d8a199"} Jul 23 00:20:52.456647 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:52.456604 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" podStartSLOduration=5.456590702 podStartE2EDuration="5.456590702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:52.455339773 +0000 UTC m=+43.761487767" watchObservedRunningTime="2026-07-23 00:20:52.456590702 +0000 UTC m=+43.762738717" Jul 23 00:20:59.461702 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.461664 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g55qm" event={"ID":"36de04af-c095-46e4-ae46-4b049fda43fc","Type":"ContainerStarted","Data":"04a9629d25585da39be94073eb4520a16eccb6bedf6ed5ce02b226326f93cee6"} Jul 23 00:20:59.462459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.461727 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-g55qm" event={"ID":"36de04af-c095-46e4-ae46-4b049fda43fc","Type":"ContainerStarted","Data":"171de2091455e44c975b3575017aecaa0abb3528b8eccf0310d201f31fd159d2"} Jul 23 00:20:59.462459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.461819 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-g55qm" Jul 23 00:20:59.463352 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.463331 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6q89s" event={"ID":"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd","Type":"ContainerStarted","Data":"92b0cc93ca9c60c49818dca5be91136c0602adbc1643f4fb994300976e747e57"} Jul 23 00:20:59.465489 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.465119 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gmhmg" event={"ID":"5cfcd5ef-d275-4d6a-b7ca-1456b86dd751","Type":"ContainerStarted","Data":"4879dd8be89909c8889ce174a768771b9921a510797dfc2bca32bb67d24b1164"} Jul 23 00:20:59.477667 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.477628 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-g55qm" podStartSLOduration=1.6510464489999999 podStartE2EDuration="8.477616874s" totalImagesPullingTime="6.826570425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:59.476963798 +0000 UTC m=+50.783111817" watchObservedRunningTime="2026-07-23 00:20:59.477616874 +0000 UTC m=+50.783764859" Jul 23 00:20:59.491311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:20:59.491273 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gmhmg" podStartSLOduration=1.639966035 podStartE2EDuration="8.491262633s" totalImagesPullingTime="6.851296598s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:20:59.490310448 +0000 UTC m=+50.796458460" watchObservedRunningTime="2026-07-23 00:20:59.491262633 +0000 UTC m=+50.797410621" Jul 23 00:21:04.485624 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.485581 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6q89s" event={"ID":"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd","Type":"ContainerStarted","Data":"6dcd2ed9bc6bcaca554b3aa40a09914b99029146549a0c5e5ccc380c3126ea3a"} Jul 23 00:21:04.796465 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.796384 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf"] Jul 23 00:21:04.799009 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.798987 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:04.801478 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.801460 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 23 00:21:04.801586 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.801502 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 23 00:21:04.802440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.802417 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 23 00:21:04.802553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.802436 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 23 00:21:04.802553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.802526 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 23 00:21:04.802675 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.802567 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-pt6k9\"" Jul 23 00:21:04.811937 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.811660 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf"] Jul 23 00:21:04.840530 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.840505 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-fv2b4"] Jul 23 00:21:04.842642 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.842614 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.845212 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.845184 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 23 00:21:04.845387 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.845359 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 23 00:21:04.845476 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.845452 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-mjp2b\"" Jul 23 00:21:04.845476 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.845466 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 23 00:21:04.898239 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898214 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-sys\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898348 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898249 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-metrics-client-ca\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898348 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:04.898348 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898309 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898348 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898334 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898353 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-root\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-wtmp\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898423 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2889d-494b-4c1f-a417-c95225c9e358-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:04.898540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898469 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjgck\" (UniqueName: \"kubernetes.io/projected/2ef2889d-494b-4c1f-a417-c95225c9e358-kube-api-access-zjgck\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:04.898540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898523 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:04.898741 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-textfile\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898741 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898576 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xrlk\" (UniqueName: \"kubernetes.io/projected/ef8a023b-bd72-443a-9653-cba071e65ec4-kube-api-access-4xrlk\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.898741 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.898611 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-accelerators-collector-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:04.999868 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999811 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-sys\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999873 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-metrics-client-ca\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999909 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999933 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999946 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-sys\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:04.999954 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000011 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-root\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000042 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-wtmp\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2889d-494b-4c1f-a417-c95225c9e358-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000099 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjgck\" (UniqueName: \"kubernetes.io/projected/2ef2889d-494b-4c1f-a417-c95225c9e358-kube-api-access-zjgck\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000136 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-root\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000151 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000191 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-textfile\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000210 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-wtmp\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000230 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xrlk\" (UniqueName: \"kubernetes.io/projected/ef8a023b-bd72-443a-9653-cba071e65ec4-kube-api-access-4xrlk\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000272 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-accelerators-collector-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:05.000336 2575 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 23 00:21:05.000458 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:05.000427 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls podName:ef8a023b-bd72-443a-9653-cba071e65ec4 nodeName:}" failed. No retries permitted until 2026-07-23 00:21:05.5004073 +0000 UTC m=+56.806555309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls") pod "node-exporter-fv2b4" (UID: "ef8a023b-bd72-443a-9653-cba071e65ec4") : secret "node-exporter-tls" not found Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:05.000610 2575 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000667 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-metrics-client-ca\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:05.000704 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls podName:2ef2889d-494b-4c1f-a417-c95225c9e358 nodeName:}" failed. No retries permitted until 2026-07-23 00:21:05.500667668 +0000 UTC m=+56.806815661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-67qkf" (UID: "2ef2889d-494b-4c1f-a417-c95225c9e358") : secret "openshift-state-metrics-tls" not found Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000733 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-accelerators-collector-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000866 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2ef2889d-494b-4c1f-a417-c95225c9e358-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.001011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.000958 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-textfile\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.005210 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.005093 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.005210 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.005160 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.010989 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.010939 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xrlk\" (UniqueName: \"kubernetes.io/projected/ef8a023b-bd72-443a-9653-cba071e65ec4-kube-api-access-4xrlk\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.011436 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.011412 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjgck\" (UniqueName: \"kubernetes.io/projected/2ef2889d-494b-4c1f-a417-c95225c9e358-kube-api-access-zjgck\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.505222 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.505190 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.505222 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.505226 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.507798 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.507748 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ef8a023b-bd72-443a-9653-cba071e65ec4-node-exporter-tls\") pod \"node-exporter-fv2b4\" (UID: \"ef8a023b-bd72-443a-9653-cba071e65ec4\") " pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:05.507798 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.507770 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2ef2889d-494b-4c1f-a417-c95225c9e358-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-67qkf\" (UID: \"2ef2889d-494b-4c1f-a417-c95225c9e358\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.710543 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.710509 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" Jul 23 00:21:05.753607 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:05.753579 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-fv2b4" Jul 23 00:21:07.020275 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:07.020231 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podef8a023b_bd72_443a_9653_cba071e65ec4.slice/crio-c11b9c4fe48dd19c87112637f2683aab3b289d5ad2e1c748b9ded9462007a2c3 WatchSource:0}: Error finding container c11b9c4fe48dd19c87112637f2683aab3b289d5ad2e1c748b9ded9462007a2c3: Status 404 returned error can't find the container with id c11b9c4fe48dd19c87112637f2683aab3b289d5ad2e1c748b9ded9462007a2c3 Jul 23 00:21:07.136277 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.136253 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf"] Jul 23 00:21:07.139176 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:07.139153 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2ef2889d_494b_4c1f_a417_c95225c9e358.slice/crio-5164c41b33d14d011ca453d533e33b89ff863e6d022665272186d5905ae46b16 WatchSource:0}: Error finding container 5164c41b33d14d011ca453d533e33b89ff863e6d022665272186d5905ae46b16: Status 404 returned error can't find the container with id 5164c41b33d14d011ca453d533e33b89ff863e6d022665272186d5905ae46b16 Jul 23 00:21:07.496623 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.496586 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" event={"ID":"2ef2889d-494b-4c1f-a417-c95225c9e358","Type":"ContainerStarted","Data":"ea24895b32a567657e94efb902ec566b264949a2f59ec5fcb5e9f2c400863f60"} Jul 23 00:21:07.496804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.496648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" event={"ID":"2ef2889d-494b-4c1f-a417-c95225c9e358","Type":"ContainerStarted","Data":"2d8a45e7655893e21e3fe2a7bcff1127efb8a0355fa955290e88920d92454ef5"} Jul 23 00:21:07.496804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.496673 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" event={"ID":"2ef2889d-494b-4c1f-a417-c95225c9e358","Type":"ContainerStarted","Data":"5164c41b33d14d011ca453d533e33b89ff863e6d022665272186d5905ae46b16"} Jul 23 00:21:07.498615 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.498586 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-6q89s" event={"ID":"a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd","Type":"ContainerStarted","Data":"265d92219bbef54d7c96cc2017b808028c893dc5be529a3100fd47b35151d2e8"} Jul 23 00:21:07.499725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.499701 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fv2b4" event={"ID":"ef8a023b-bd72-443a-9653-cba071e65ec4","Type":"ContainerStarted","Data":"c11b9c4fe48dd19c87112637f2683aab3b289d5ad2e1c748b9ded9462007a2c3"} Jul 23 00:21:07.529970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.529923 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-6q89s" podStartSLOduration=1.892348745 podStartE2EDuration="16.529911853s" totalImagesPullingTime="14.637563108s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:51 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:07.527907649 +0000 UTC m=+58.834055655" watchObservedRunningTime="2026-07-23 00:21:07.529911853 +0000 UTC m=+58.836059874" Jul 23 00:21:07.789524 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.789451 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7565bff569-k78sx"] Jul 23 00:21:07.792760 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.792735 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.795338 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795310 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 23 00:21:07.795472 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795430 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-lnr64\"" Jul 23 00:21:07.795524 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795310 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-c5qnml3b6du8h\"" Jul 23 00:21:07.795583 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795567 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 23 00:21:07.795642 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795502 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 23 00:21:07.795732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795713 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 23 00:21:07.795992 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.795789 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 23 00:21:07.805689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.805616 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7565bff569-k78sx"] Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825214 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-grpc-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825308 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825394 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825421 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825519 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-27n47\" (UniqueName: \"kubernetes.io/projected/87252b26-97dc-4e17-8f1b-6ae2d907b90c-kube-api-access-27n47\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825553 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87252b26-97dc-4e17-8f1b-6ae2d907b90c-metrics-client-ca\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.825707 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825600 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.826178 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.825708 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927291 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927254 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927442 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927318 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927442 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927383 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-grpc-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927438 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927463 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927486 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927515 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-27n47\" (UniqueName: \"kubernetes.io/projected/87252b26-97dc-4e17-8f1b-6ae2d907b90c-kube-api-access-27n47\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.927696 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.927541 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87252b26-97dc-4e17-8f1b-6ae2d907b90c-metrics-client-ca\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.928303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.928273 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/87252b26-97dc-4e17-8f1b-6ae2d907b90c-metrics-client-ca\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.930467 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.930423 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.930571 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.930548 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.930691 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.930663 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.930920 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.930895 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.931119 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.931101 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.931989 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.931972 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/87252b26-97dc-4e17-8f1b-6ae2d907b90c-secret-grpc-tls\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:07.935910 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:07.935892 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-27n47\" (UniqueName: \"kubernetes.io/projected/87252b26-97dc-4e17-8f1b-6ae2d907b90c-kube-api-access-27n47\") pod \"thanos-querier-7565bff569-k78sx\" (UID: \"87252b26-97dc-4e17-8f1b-6ae2d907b90c\") " pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:08.104989 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.104958 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:08.242484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.242458 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7565bff569-k78sx"] Jul 23 00:21:08.352313 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:08.352284 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87252b26_97dc_4e17_8f1b_6ae2d907b90c.slice/crio-16e087c4363d8c2c676545fb31e8bc912bb9bf1e105b262e1d7160ffad1463a2 WatchSource:0}: Error finding container 16e087c4363d8c2c676545fb31e8bc912bb9bf1e105b262e1d7160ffad1463a2: Status 404 returned error can't find the container with id 16e087c4363d8c2c676545fb31e8bc912bb9bf1e105b262e1d7160ffad1463a2 Jul 23 00:21:08.504336 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.504301 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"16e087c4363d8c2c676545fb31e8bc912bb9bf1e105b262e1d7160ffad1463a2"} Jul 23 00:21:08.505929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.505902 2575 generic.go:356] "Generic (PLEG): container finished" podID="ef8a023b-bd72-443a-9653-cba071e65ec4" containerID="2e4f6954244a788c86f2734b96f01f9341a97a175dd4e0cd34eb32d9e77fcaa9" exitCode=0 Jul 23 00:21:08.506037 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.505990 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fv2b4" event={"ID":"ef8a023b-bd72-443a-9653-cba071e65ec4","Type":"ContainerDied","Data":"2e4f6954244a788c86f2734b96f01f9341a97a175dd4e0cd34eb32d9e77fcaa9"} Jul 23 00:21:08.508305 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.508270 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" event={"ID":"2ef2889d-494b-4c1f-a417-c95225c9e358","Type":"ContainerStarted","Data":"48b5a3111d9b4d141af1ad1ede81431f7915f84865f3a48c30dda4ce7a2759f2"} Jul 23 00:21:08.540577 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:08.540540 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-67qkf" podStartSLOduration=3.399085381 podStartE2EDuration="4.540527313s" totalImagesPullingTime="1.141441932s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:08.540142928 +0000 UTC m=+59.846290924" watchObservedRunningTime="2026-07-23 00:21:08.540527313 +0000 UTC m=+59.846675306" Jul 23 00:21:09.471120 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.471090 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-g55qm" Jul 23 00:21:09.514614 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.514578 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fv2b4" event={"ID":"ef8a023b-bd72-443a-9653-cba071e65ec4","Type":"ContainerStarted","Data":"fca4cdc51c56ba284692cad3b92cf5de4e0f5ce4e08db3a6ecf2380667eda48a"} Jul 23 00:21:09.514785 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.514643 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-fv2b4" event={"ID":"ef8a023b-bd72-443a-9653-cba071e65ec4","Type":"ContainerStarted","Data":"e77d1d3a91b54db65ec0c6ab65f72f401d8d2618edd5719f31ac9c6feb3f842b"} Jul 23 00:21:09.532756 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.532700 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-fv2b4" podStartSLOduration=4.504529816 podStartE2EDuration="5.532682332s" totalImagesPullingTime="963.589745ms" totalInitContainerRuntime="64.562771ms" isStatefulPod=true podCreationTimestamp="2026-07-23 00:21:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:09.532163054 +0000 UTC m=+60.838311079" watchObservedRunningTime="2026-07-23 00:21:09.532682332 +0000 UTC m=+60.838830326" Jul 23 00:21:09.603880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.603840 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2"] Jul 23 00:21:09.607069 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.607048 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.609720 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.609690 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 23 00:21:09.609720 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.609706 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-6xvwx\"" Jul 23 00:21:09.614077 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.614054 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2"] Jul 23 00:21:09.641273 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.641218 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9frn\" (UniqueName: \"kubernetes.io/projected/e81ed6d1-5380-41de-be29-e1a7988c27a5-kube-api-access-z9frn\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.641391 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.641373 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.742303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.742233 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9frn\" (UniqueName: \"kubernetes.io/projected/e81ed6d1-5380-41de-be29-e1a7988c27a5-kube-api-access-z9frn\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.742303 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.742275 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.742484 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:09.742408 2575 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 23 00:21:09.742484 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:09.742473 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert podName:e81ed6d1-5380-41de-be29-e1a7988c27a5 nodeName:}" failed. No retries permitted until 2026-07-23 00:21:10.24245236 +0000 UTC m=+61.548600348 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-tgbb2" (UID: "e81ed6d1-5380-41de-be29-e1a7988c27a5") : secret "monitoring-plugin-cert" not found Jul 23 00:21:09.750733 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.750709 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9frn\" (UniqueName: \"kubernetes.io/projected/e81ed6d1-5380-41de-be29-e1a7988c27a5-kube-api-access-z9frn\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:09.870587 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.870562 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:09.882252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.882227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.883799 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.883774 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:09.884787 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.884763 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jjlcf\"" Jul 23 00:21:09.885155 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885135 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 00:21:09.885245 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885193 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 00:21:09.885296 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885245 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 00:21:09.885345 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885320 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 00:21:09.885345 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885337 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 00:21:09.885502 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885483 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 00:21:09.885869 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.885851 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 00:21:09.890841 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.890805 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 00:21:09.943641 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943714 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943658 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943714 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943680 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943716 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943766 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:09.943944 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:09.943819 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptrvs\" (UniqueName: \"kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.010646 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.010595 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9"] Jul 23 00:21:10.017807 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.017784 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.022867 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.022850 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 23 00:21:10.023043 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.023026 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 23 00:21:10.023720 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.023608 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 23 00:21:10.024795 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.024048 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 23 00:21:10.024795 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.024289 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-hpdhd\"" Jul 23 00:21:10.024795 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.024636 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 23 00:21:10.032058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.032031 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 23 00:21:10.032169 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.032092 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9"] Jul 23 00:21:10.044484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.044766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044504 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.044766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.044766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.044766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044624 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.045122 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.044890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptrvs\" (UniqueName: \"kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.045338 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.045313 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.046171 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.046081 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.046171 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.046090 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.046171 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.046124 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.046621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.046597 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.047842 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.047804 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.048787 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.048767 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.052395 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.052376 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptrvs\" (UniqueName: \"kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs\") pod \"console-7b477cc548-gsqps\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.145944 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.145902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddptn\" (UniqueName: \"kubernetes.io/projected/83d083a1-f8e1-4851-8271-d9f6ea276e19-kube-api-access-ddptn\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146110 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.145983 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-federate-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146110 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146074 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-metrics-client-ca\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146143 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146197 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146324 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146240 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146324 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146277 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-serving-certs-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.146409 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.146346 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.193913 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.193876 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:10.247226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247200 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-metrics-client-ca\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247261 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:10.247340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247282 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247461 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-serving-certs-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247588 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247559 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247644 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247604 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddptn\" (UniqueName: \"kubernetes.io/projected/83d083a1-f8e1-4851-8271-d9f6ea276e19-kube-api-access-ddptn\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.247696 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.247648 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-federate-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.248267 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.248194 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-metrics-client-ca\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.248766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.248340 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-trusted-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.248932 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.248881 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/83d083a1-f8e1-4851-8271-d9f6ea276e19-serving-certs-ca-bundle\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.250356 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.250331 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.250450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.250418 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/e81ed6d1-5380-41de-be29-e1a7988c27a5-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-tgbb2\" (UID: \"e81ed6d1-5380-41de-be29-e1a7988c27a5\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:10.250450 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.250427 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-telemeter-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.250595 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.250573 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-secret-telemeter-client\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.250664 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.250594 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/83d083a1-f8e1-4851-8271-d9f6ea276e19-federate-client-tls\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.255770 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.255751 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddptn\" (UniqueName: \"kubernetes.io/projected/83d083a1-f8e1-4851-8271-d9f6ea276e19-kube-api-access-ddptn\") pod \"telemeter-client-d5fcfd56f-5dwq9\" (UID: \"83d083a1-f8e1-4851-8271-d9f6ea276e19\") " pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.331437 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.331401 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" Jul 23 00:21:10.405978 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.405951 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-fhw25" Jul 23 00:21:10.516409 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.516375 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:10.785218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:10.785184 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:10.788794 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:10.788767 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5879bb77_dd77_4689_8d5b_2c8208796d15.slice/crio-4d9a26bab5594dd29554bfbc755c6db37d6add33378e199809b517fac7000396 WatchSource:0}: Error finding container 4d9a26bab5594dd29554bfbc755c6db37d6add33378e199809b517fac7000396: Status 404 returned error can't find the container with id 4d9a26bab5594dd29554bfbc755c6db37d6add33378e199809b517fac7000396 Jul 23 00:21:11.011044 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.011017 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2"] Jul 23 00:21:11.014508 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:11.014474 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode81ed6d1_5380_41de_be29_e1a7988c27a5.slice/crio-2c23bc61677aba703c54d0c9c8cfd5235e27cd8798f9b0611161a45e799d44a1 WatchSource:0}: Error finding container 2c23bc61677aba703c54d0c9c8cfd5235e27cd8798f9b0611161a45e799d44a1: Status 404 returned error can't find the container with id 2c23bc61677aba703c54d0c9c8cfd5235e27cd8798f9b0611161a45e799d44a1 Jul 23 00:21:11.016378 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.016362 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9"] Jul 23 00:21:11.017690 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:11.017665 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod83d083a1_f8e1_4851_8271_d9f6ea276e19.slice/crio-877bb4ef01847feb34430d722a921c0d04d98e726bd575f817c7e6e4978a4c4e WatchSource:0}: Error finding container 877bb4ef01847feb34430d722a921c0d04d98e726bd575f817c7e6e4978a4c4e: Status 404 returned error can't find the container with id 877bb4ef01847feb34430d722a921c0d04d98e726bd575f817c7e6e4978a4c4e Jul 23 00:21:11.527102 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.527009 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"e55626a5c5dffd263cbfd5387309c17875537a6910313856ab95783f46574c7e"} Jul 23 00:21:11.527553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.527122 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"08bff8cdd445f9ed48c4695bd3008f42438410d4ab4c5f8d15ad7483bb329e12"} Jul 23 00:21:11.527553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.527165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"8135fc80b96c7300eea85e7bb5be75d0088546a942376b32639884f60de7399e"} Jul 23 00:21:11.528434 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.528393 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b477cc548-gsqps" event={"ID":"5879bb77-dd77-4689-8d5b-2c8208796d15","Type":"ContainerStarted","Data":"4d9a26bab5594dd29554bfbc755c6db37d6add33378e199809b517fac7000396"} Jul 23 00:21:11.529668 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.529629 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" event={"ID":"e81ed6d1-5380-41de-be29-e1a7988c27a5","Type":"ContainerStarted","Data":"2c23bc61677aba703c54d0c9c8cfd5235e27cd8798f9b0611161a45e799d44a1"} Jul 23 00:21:11.530928 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:11.530903 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" event={"ID":"83d083a1-f8e1-4851-8271-d9f6ea276e19","Type":"ContainerStarted","Data":"877bb4ef01847feb34430d722a921c0d04d98e726bd575f817c7e6e4978a4c4e"} Jul 23 00:21:13.447272 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.447241 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5bd67c9c9f-44p27" Jul 23 00:21:13.541930 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.541892 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"e491379ec443f3a9d49124ad1c68569f1ad2ba96d34e1eaaf4cb5bac1ba25b5c"} Jul 23 00:21:13.543561 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.543536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" event={"ID":"e81ed6d1-5380-41de-be29-e1a7988c27a5","Type":"ContainerStarted","Data":"6a89bc30d99c20fc4ea1353681a7f2a266abfa6749bf91e162e5b59949ac7a62"} Jul 23 00:21:13.543791 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.543727 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:13.549335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.549317 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" Jul 23 00:21:13.559192 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:13.559148 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-tgbb2" podStartSLOduration=2.884049762 podStartE2EDuration="4.559131962s" totalImagesPullingTime="1.6750822s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:13.558710619 +0000 UTC m=+64.864858614" watchObservedRunningTime="2026-07-23 00:21:13.559131962 +0000 UTC m=+64.865279958" Jul 23 00:21:14.550421 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.550340 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"d83553eacd0934d38205f1cda2221a77a1cc42e2c9c97d3daacf69a027d3fed9"} Jul 23 00:21:14.550797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.550460 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" event={"ID":"87252b26-97dc-4e17-8f1b-6ae2d907b90c","Type":"ContainerStarted","Data":"1bb5e4f996a41744ed0888754f421d1693503fe6c11f770b350c232ea7f795ea"} Jul 23 00:21:14.550797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.550514 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:14.551967 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.551934 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b477cc548-gsqps" event={"ID":"5879bb77-dd77-4689-8d5b-2c8208796d15","Type":"ContainerStarted","Data":"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928"} Jul 23 00:21:14.553289 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.553267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" event={"ID":"83d083a1-f8e1-4851-8271-d9f6ea276e19","Type":"ContainerStarted","Data":"7c59b0d7afdf3c3d1aba8a631bdf86d3af2c172c9d26d2c9434e5f97c8d6cf4f"} Jul 23 00:21:14.574196 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.574145 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" podStartSLOduration=3.557330924 podStartE2EDuration="7.574132234s" totalImagesPullingTime="4.01680131s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:21:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:14.573559775 +0000 UTC m=+65.879707789" watchObservedRunningTime="2026-07-23 00:21:14.574132234 +0000 UTC m=+65.880280227" Jul 23 00:21:14.592633 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.592592 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7b477cc548-gsqps" podStartSLOduration=2.14466465 podStartE2EDuration="5.592579329s" totalImagesPullingTime="3.447914679s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:21:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:14.592015835 +0000 UTC m=+65.898163848" watchObservedRunningTime="2026-07-23 00:21:14.592579329 +0000 UTC m=+65.898727323" Jul 23 00:21:14.888648 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.888620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:21:14.891195 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.891179 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 23 00:21:14.901219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.901201 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/22e1b738-7df0-4e85-99a9-335b1abd8c66-original-pull-secret\") pod \"global-pull-secret-syncer-htfz9\" (UID: \"22e1b738-7df0-4e85-99a9-335b1abd8c66\") " pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:21:14.989738 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.989711 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:21:14.992128 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:14.992112 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 23 00:21:15.002061 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.002040 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/562817ed-d6e5-427e-9acf-d9e3804bd86c-metrics-certs\") pod \"network-metrics-daemon-krcjb\" (UID: \"562817ed-d6e5-427e-9acf-d9e3804bd86c\") " pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:21:15.049837 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.049815 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-52cgs\"" Jul 23 00:21:15.052850 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.052820 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-htfz9" Jul 23 00:21:15.057899 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.057881 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-krcjb" Jul 23 00:21:15.090318 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.090290 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:21:15.092897 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.092874 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 23 00:21:15.104088 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.103872 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 23 00:21:15.114409 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.114383 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5tl2m\" (UniqueName: \"kubernetes.io/projected/30602cc0-76d5-4e02-b766-9b522b8c7b96-kube-api-access-5tl2m\") pod \"network-check-target-945mz\" (UID: \"30602cc0-76d5-4e02-b766-9b522b8c7b96\") " pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:21:15.175547 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.175515 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-htfz9"] Jul 23 00:21:15.179064 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:15.179035 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22e1b738_7df0_4e85_99a9_335b1abd8c66.slice/crio-be7b36d243f744572ad62e9d003bf1206fb28827e82c0abad018ced27decdf9b WatchSource:0}: Error finding container be7b36d243f744572ad62e9d003bf1206fb28827e82c0abad018ced27decdf9b: Status 404 returned error can't find the container with id be7b36d243f744572ad62e9d003bf1206fb28827e82c0abad018ced27decdf9b Jul 23 00:21:15.198785 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.198765 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-krcjb"] Jul 23 00:21:15.201369 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:15.201341 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod562817ed_d6e5_427e_9acf_d9e3804bd86c.slice/crio-24744b1552c40fb1c09ada99377f25a195feff73bea48a9f73cf1c5181338807 WatchSource:0}: Error finding container 24744b1552c40fb1c09ada99377f25a195feff73bea48a9f73cf1c5181338807: Status 404 returned error can't find the container with id 24744b1552c40fb1c09ada99377f25a195feff73bea48a9f73cf1c5181338807 Jul 23 00:21:15.345039 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.345017 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2ns5k\"" Jul 23 00:21:15.353060 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.353031 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:21:15.471047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.471019 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-945mz"] Jul 23 00:21:15.474026 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:21:15.473993 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30602cc0_76d5_4e02_b766_9b522b8c7b96.slice/crio-19b4e758f2144de1ed55b28410029c97f4f8b02ade40bcff5645947e260025a4 WatchSource:0}: Error finding container 19b4e758f2144de1ed55b28410029c97f4f8b02ade40bcff5645947e260025a4: Status 404 returned error can't find the container with id 19b4e758f2144de1ed55b28410029c97f4f8b02ade40bcff5645947e260025a4 Jul 23 00:21:15.557750 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.557713 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-945mz" event={"ID":"30602cc0-76d5-4e02-b766-9b522b8c7b96","Type":"ContainerStarted","Data":"19b4e758f2144de1ed55b28410029c97f4f8b02ade40bcff5645947e260025a4"} Jul 23 00:21:15.558856 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.558813 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-htfz9" event={"ID":"22e1b738-7df0-4e85-99a9-335b1abd8c66","Type":"ContainerStarted","Data":"be7b36d243f744572ad62e9d003bf1206fb28827e82c0abad018ced27decdf9b"} Jul 23 00:21:15.559866 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:15.559837 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krcjb" event={"ID":"562817ed-d6e5-427e-9acf-d9e3804bd86c","Type":"ContainerStarted","Data":"24744b1552c40fb1c09ada99377f25a195feff73bea48a9f73cf1c5181338807"} Jul 23 00:21:17.571870 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.571816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krcjb" event={"ID":"562817ed-d6e5-427e-9acf-d9e3804bd86c","Type":"ContainerStarted","Data":"656c13b85d4307581b40adabab6cef75dad0a8f25010f85862f625e5901ebfb8"} Jul 23 00:21:17.572328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.571902 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-krcjb" event={"ID":"562817ed-d6e5-427e-9acf-d9e3804bd86c","Type":"ContainerStarted","Data":"8470a7c184f03370b8fe6a591b03afb1757a82bf9117f0f2e3f512fdceafe794"} Jul 23 00:21:17.574559 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.574531 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" event={"ID":"83d083a1-f8e1-4851-8271-d9f6ea276e19","Type":"ContainerStarted","Data":"1f840b83b2cfd13e662351e7e9dc3290ad546ce836cdc1e51640942b28b797cb"} Jul 23 00:21:17.574697 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.574588 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" event={"ID":"83d083a1-f8e1-4851-8271-d9f6ea276e19","Type":"ContainerStarted","Data":"b3f740b647c837d3a0556afc285f5c72ac81b637d9f25660c0fdd0429f59a2a9"} Jul 23 00:21:17.588715 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.588653 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-krcjb" podStartSLOduration=66.15629676 podStartE2EDuration="1m7.588634169s" totalImagesPullingTime="1.432337409s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:17.588193764 +0000 UTC m=+68.894341768" watchObservedRunningTime="2026-07-23 00:21:17.588634169 +0000 UTC m=+68.894782160" Jul 23 00:21:17.615392 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:17.615342 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-d5fcfd56f-5dwq9" podStartSLOduration=3.069886272 podStartE2EDuration="8.615326735s" totalImagesPullingTime="5.545440463s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:21:09 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:17.611229567 +0000 UTC m=+68.917377600" watchObservedRunningTime="2026-07-23 00:21:17.615326735 +0000 UTC m=+68.921474730" Jul 23 00:21:18.279805 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:18.279762 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:20.194575 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.194521 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:20.566075 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.566043 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7565bff569-k78sx" Jul 23 00:21:20.588202 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.588167 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-945mz" event={"ID":"30602cc0-76d5-4e02-b766-9b522b8c7b96","Type":"ContainerStarted","Data":"1dc138dc5daf947eae32cf8d27fd522679cca01c7969e2e7b047d26f1ba3d5c8"} Jul 23 00:21:20.588369 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.588226 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:21:20.589668 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.589641 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-htfz9" event={"ID":"22e1b738-7df0-4e85-99a9-335b1abd8c66","Type":"ContainerStarted","Data":"7e2de2eaf439b4f43525fdda304b4b2fe0255370e7d51ede9e7c6cd0f05f7796"} Jul 23 00:21:20.614649 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.614603 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-945mz" podStartSLOduration=65.927442681 podStartE2EDuration="1m10.614589758s" totalImagesPullingTime="4.687147077s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:20.614237785 +0000 UTC m=+71.920385808" watchObservedRunningTime="2026-07-23 00:21:20.614589758 +0000 UTC m=+71.920737751" Jul 23 00:21:20.628425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:20.628345 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-htfz9" podStartSLOduration=65.64116915 podStartE2EDuration="1m10.628326046s" totalImagesPullingTime="4.987156896s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 00:20:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:21:20.627779938 +0000 UTC m=+71.933927954" watchObservedRunningTime="2026-07-23 00:21:20.628326046 +0000 UTC m=+71.934474043" Jul 23 00:21:43.301419 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.301282 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7b477cc548-gsqps" podUID="5879bb77-dd77-4689-8d5b-2c8208796d15" containerName="console" containerID="cri-o://817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928" gracePeriod=15 Jul 23 00:21:43.555939 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.555888 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:43.677047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.677016 2575 generic.go:356] "Generic (PLEG): container finished" podID="5879bb77-dd77-4689-8d5b-2c8208796d15" containerID="817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928" exitCode=0 Jul 23 00:21:43.677186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.677056 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7b477cc548-gsqps" Jul 23 00:21:43.677186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.677067 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b477cc548-gsqps" event={"ID":"5879bb77-dd77-4689-8d5b-2c8208796d15","Type":"ContainerDied","Data":"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928"} Jul 23 00:21:43.677186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.677114 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7b477cc548-gsqps" event={"ID":"5879bb77-dd77-4689-8d5b-2c8208796d15","Type":"ContainerDied","Data":"4d9a26bab5594dd29554bfbc755c6db37d6add33378e199809b517fac7000396"} Jul 23 00:21:43.677186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.677135 2575 scope.go:117] "RemoveContainer" containerID="817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928" Jul 23 00:21:43.684984 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.684967 2575 scope.go:117] "RemoveContainer" containerID="817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928" Jul 23 00:21:43.685235 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:21:43.685215 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928\": container with ID starting with 817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928 not found: ID does not exist" containerID="817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928" Jul 23 00:21:43.685305 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.685247 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928"} err="failed to get container status \"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928\": rpc error: code = NotFound desc = could not find container \"817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928\": container with ID starting with 817818367e1c8be46ca2f141fcb44475157fa99baab1772d196e369ecd271928 not found: ID does not exist" Jul 23 00:21:43.691482 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691465 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691536 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691503 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691536 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691524 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691634 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691542 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ptrvs\" (UniqueName: \"kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691634 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691565 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691634 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691594 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.691771 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691637 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert\") pod \"5879bb77-dd77-4689-8d5b-2c8208796d15\" (UID: \"5879bb77-dd77-4689-8d5b-2c8208796d15\") " Jul 23 00:21:43.692021 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691986 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:21:43.692134 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.691999 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca" (OuterVolumeSpecName: "service-ca") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:21:43.692134 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.692002 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config" (OuterVolumeSpecName: "console-config") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:21:43.692134 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.692054 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:21:43.693783 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.693753 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:21:43.693886 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.693823 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:21:43.693932 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.693882 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs" (OuterVolumeSpecName: "kube-api-access-ptrvs") pod "5879bb77-dd77-4689-8d5b-2c8208796d15" (UID: "5879bb77-dd77-4689-8d5b-2c8208796d15"). InnerVolumeSpecName "kube-api-access-ptrvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:21:43.792824 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792800 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792824 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792821 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792852 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792865 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5879bb77-dd77-4689-8d5b-2c8208796d15-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792875 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792883 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5879bb77-dd77-4689-8d5b-2c8208796d15-trusted-ca-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.792947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.792891 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ptrvs\" (UniqueName: \"kubernetes.io/projected/5879bb77-dd77-4689-8d5b-2c8208796d15-kube-api-access-ptrvs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:21:43.997841 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:43.997811 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:44.001603 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:44.001581 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7b477cc548-gsqps"] Jul 23 00:21:45.235044 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:45.235003 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5879bb77-dd77-4689-8d5b-2c8208796d15" path="/var/lib/kubelet/pods/5879bb77-dd77-4689-8d5b-2c8208796d15/volumes" Jul 23 00:21:51.600257 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:21:51.600228 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-945mz" Jul 23 00:22:25.229305 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.229270 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:22:25.229857 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.229555 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5879bb77-dd77-4689-8d5b-2c8208796d15" containerName="console" Jul 23 00:22:25.229857 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.229568 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5879bb77-dd77-4689-8d5b-2c8208796d15" containerName="console" Jul 23 00:22:25.229857 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.229631 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5879bb77-dd77-4689-8d5b-2c8208796d15" containerName="console" Jul 23 00:22:25.232357 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.232330 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.234766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.234744 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 23 00:22:25.235275 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.235255 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 23 00:22:25.238265 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238242 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 23 00:22:25.238381 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238359 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 23 00:22:25.238471 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238456 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 23 00:22:25.238735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238715 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 23 00:22:25.238858 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238783 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 23 00:22:25.238925 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.238863 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-jjlcf\"" Jul 23 00:22:25.242929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.242907 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 23 00:22:25.246448 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.246425 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:22:25.266922 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.266895 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9stq\" (UniqueName: \"kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.266926 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.266969 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267011 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.267003 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267127 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.267059 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267127 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.267107 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.267198 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.267126 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368350 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368325 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368351 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368379 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9stq\" (UniqueName: \"kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368418 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368565 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368453 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368565 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368491 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.368565 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.368528 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.369167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.369140 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.369265 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.369219 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.369313 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.369267 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.369513 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.369491 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.370880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.370861 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.370990 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.370973 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.376483 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.376462 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9stq\" (UniqueName: \"kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq\") pod \"console-768cff7b7b-h677w\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.544644 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.544585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:25.665227 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.665203 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:22:25.667793 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:22:25.667765 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16896afd_31d0_48f0_959b_91b5464857dd.slice/crio-736f1aec04ee2767e2c1c2619a08fe5a6e326c16b2cd3fed00b401ff03379980 WatchSource:0}: Error finding container 736f1aec04ee2767e2c1c2619a08fe5a6e326c16b2cd3fed00b401ff03379980: Status 404 returned error can't find the container with id 736f1aec04ee2767e2c1c2619a08fe5a6e326c16b2cd3fed00b401ff03379980 Jul 23 00:22:25.826808 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.826771 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768cff7b7b-h677w" event={"ID":"16896afd-31d0-48f0-959b-91b5464857dd","Type":"ContainerStarted","Data":"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b"} Jul 23 00:22:25.826973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.826820 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768cff7b7b-h677w" event={"ID":"16896afd-31d0-48f0-959b-91b5464857dd","Type":"ContainerStarted","Data":"736f1aec04ee2767e2c1c2619a08fe5a6e326c16b2cd3fed00b401ff03379980"} Jul 23 00:22:25.844056 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:25.844013 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-768cff7b7b-h677w" podStartSLOduration=0.843996639 podStartE2EDuration="843.996639ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:22:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:22:25.842635163 +0000 UTC m=+137.148783178" watchObservedRunningTime="2026-07-23 00:22:25.843996639 +0000 UTC m=+137.150144673" Jul 23 00:22:26.852336 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.852300 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 00:22:26.855026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.855009 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.857856 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.857814 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-k6vvj\"" Jul 23 00:22:26.858013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.857933 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 23 00:22:26.858013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.857991 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 23 00:22:26.858013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.857933 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 23 00:22:26.858182 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.858118 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 23 00:22:26.858182 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.858129 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 23 00:22:26.858315 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.858297 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 23 00:22:26.858376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.858368 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 23 00:22:26.858427 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.858388 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 23 00:22:26.863587 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.863566 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 23 00:22:26.868146 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.868127 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 00:22:26.878156 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878135 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-web-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878231 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878175 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-config-volume\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878231 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878231 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878223 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-config-out\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878389 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878374 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878400 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878457 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878423 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878457 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878438 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878455 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878481 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878498 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5zbl\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-kube-api-access-b5zbl\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878606 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.878662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.878637 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.979998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.979961 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.979998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.979999 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980026 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-web-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-config-volume\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980065 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980079 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-config-out\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980103 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980122 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980140 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980164 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980188 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980219 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980223 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980671 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5zbl\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-kube-api-access-b5zbl\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.980815 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980788 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.981013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.980947 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983382 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.982920 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7bff247c-874d-49f0-a1a6-5333b681f321-config-out\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983382 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983249 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7bff247c-874d-49f0-a1a6-5333b681f321-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983382 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983278 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983600 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983433 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-web-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983717 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983691 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983782 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983721 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.983869 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.983784 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.984225 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.984204 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-config-volume\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.984588 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.984568 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.985150 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.985129 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7bff247c-874d-49f0-a1a6-5333b681f321-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:26.988548 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:26.988525 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5zbl\" (UniqueName: \"kubernetes.io/projected/7bff247c-874d-49f0-a1a6-5333b681f321-kube-api-access-b5zbl\") pod \"alertmanager-main-0\" (UID: \"7bff247c-874d-49f0-a1a6-5333b681f321\") " pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:27.165139 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:27.165056 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 23 00:22:27.295455 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:27.295430 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 23 00:22:27.297268 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:22:27.297240 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bff247c_874d_49f0_a1a6_5333b681f321.slice/crio-3b79d1863ce5ed1abace59806567233039acf974d35e082ce7984402126a0d11 WatchSource:0}: Error finding container 3b79d1863ce5ed1abace59806567233039acf974d35e082ce7984402126a0d11: Status 404 returned error can't find the container with id 3b79d1863ce5ed1abace59806567233039acf974d35e082ce7984402126a0d11 Jul 23 00:22:27.835510 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:27.835477 2575 generic.go:356] "Generic (PLEG): container finished" podID="7bff247c-874d-49f0-a1a6-5333b681f321" containerID="a1b569d13d8cd78bf73f8cc0c964c4f15ca7e66e8aae1ca5b1085b39721c27a3" exitCode=0 Jul 23 00:22:27.835662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:27.835547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerDied","Data":"a1b569d13d8cd78bf73f8cc0c964c4f15ca7e66e8aae1ca5b1085b39721c27a3"} Jul 23 00:22:27.835662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:27.835613 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"3b79d1863ce5ed1abace59806567233039acf974d35e082ce7984402126a0d11"} Jul 23 00:22:29.845528 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845448 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"9b666779ad202b97eb5ae98754ceaf54805672bba1b424a7484433269705cac6"} Jul 23 00:22:29.845528 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845513 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"e40a542df0ac9dd997573df0b16e35060dde0a1f81cfde4518ae87047bee9a4d"} Jul 23 00:22:29.845907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845546 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"5c776fa33530d924f7be23e8d5e5802197e240b9b127f16e0370b0071c611d1c"} Jul 23 00:22:29.845907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845571 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"58a1b3c39fb5dedf85583a821b925959af2c19430c17846319c5006009117ca4"} Jul 23 00:22:29.845907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845600 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"d1dc7d652c67e279ed47c98acb7a6425c663d1d0ff11c7491e26c20b200866cc"} Jul 23 00:22:29.845907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.845623 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7bff247c-874d-49f0-a1a6-5333b681f321","Type":"ContainerStarted","Data":"b550fa60a88c269ea72b253d9a6b7c4f5a3d2364382b1578fbe56c987d5ae2c0"} Jul 23 00:22:29.878343 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:29.878295 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.383211867 podStartE2EDuration="3.878280504s" totalImagesPullingTime="1.479830741s" totalInitContainerRuntime="15.237896ms" isStatefulPod=false podCreationTimestamp="2026-07-23 00:22:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:22:29.877302303 +0000 UTC m=+141.183450309" watchObservedRunningTime="2026-07-23 00:22:29.878280504 +0000 UTC m=+141.184428540" Jul 23 00:22:35.544851 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:35.544789 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:35.544851 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:35.544860 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:35.549980 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:35.549956 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:22:35.875676 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:22:35.875644 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:23:07.754088 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.754008 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz"] Jul 23 00:23:07.756538 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.756521 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.759265 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.759241 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 00:23:07.759370 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.759260 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 00:23:07.760417 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.760396 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2d2n8\"" Jul 23 00:23:07.766583 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.766559 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz"] Jul 23 00:23:07.870958 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.870927 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.871062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.870963 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.871062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.871033 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsmj7\" (UniqueName: \"kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.971626 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.971595 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.971745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.971636 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.971745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.971695 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsmj7\" (UniqueName: \"kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.971983 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.971962 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.972059 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.971995 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:07.979686 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:07.979669 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsmj7\" (UniqueName: \"kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:08.066920 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:08.066902 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:08.193104 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:08.193073 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz"] Jul 23 00:23:08.195390 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:23:08.195364 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc25d1e91_b3d4_46d4_97a0_8a8454f1cb55.slice/crio-a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a WatchSource:0}: Error finding container a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a: Status 404 returned error can't find the container with id a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a Jul 23 00:23:08.996166 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:08.996130 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" event={"ID":"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55","Type":"ContainerStarted","Data":"a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a"} Jul 23 00:23:16.025064 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:16.024984 2575 generic.go:356] "Generic (PLEG): container finished" podID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerID="a443cd81eaa27bcc7bec37670b5b0df2a18f7223561d403237e338dc90c73cc0" exitCode=0 Jul 23 00:23:16.025413 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:16.025074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" event={"ID":"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55","Type":"ContainerDied","Data":"a443cd81eaa27bcc7bec37670b5b0df2a18f7223561d403237e338dc90c73cc0"} Jul 23 00:23:19.038628 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:19.038593 2575 generic.go:356] "Generic (PLEG): container finished" podID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerID="7cf72ae90051feb45b28c8fb746a4fcc2dd38c3eab1406c614b116c357c4bdcc" exitCode=0 Jul 23 00:23:19.039013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:19.038654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" event={"ID":"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55","Type":"ContainerDied","Data":"7cf72ae90051feb45b28c8fb746a4fcc2dd38c3eab1406c614b116c357c4bdcc"} Jul 23 00:23:32.088819 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:32.088784 2575 generic.go:356] "Generic (PLEG): container finished" podID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerID="296a4f9b50ca38eb896c8b6f70cbd04865df39947dc4ac5458e6288884c334af" exitCode=0 Jul 23 00:23:32.089240 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:32.088873 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" event={"ID":"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55","Type":"ContainerDied","Data":"296a4f9b50ca38eb896c8b6f70cbd04865df39947dc4ac5458e6288884c334af"} Jul 23 00:23:33.223001 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.222982 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:33.374712 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.374633 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util\") pod \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " Jul 23 00:23:33.374712 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.374670 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle\") pod \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " Jul 23 00:23:33.374899 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.374725 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsmj7\" (UniqueName: \"kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7\") pod \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\" (UID: \"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55\") " Jul 23 00:23:33.375453 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.375422 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle" (OuterVolumeSpecName: "bundle") pod "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" (UID: "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:23:33.376934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.376898 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7" (OuterVolumeSpecName: "kube-api-access-nsmj7") pod "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" (UID: "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55"). InnerVolumeSpecName "kube-api-access-nsmj7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:23:33.378873 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.378854 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util" (OuterVolumeSpecName: "util") pod "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" (UID: "c25d1e91-b3d4-46d4-97a0-8a8454f1cb55"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:23:33.475627 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.475604 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:33.475627 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.475627 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:33.475771 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:33.475637 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsmj7\" (UniqueName: \"kubernetes.io/projected/c25d1e91-b3d4-46d4-97a0-8a8454f1cb55-kube-api-access-nsmj7\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:34.099112 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:34.099074 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" event={"ID":"c25d1e91-b3d4-46d4-97a0-8a8454f1cb55","Type":"ContainerDied","Data":"a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a"} Jul 23 00:23:34.099268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:34.099122 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a9dd6f1cb443ad7198d6cc0b916e89551730f852f3c25b5f475c0be0ad25fb2a" Jul 23 00:23:34.099268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:34.099173 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540zgrtz" Jul 23 00:23:40.781238 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781194 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4"] Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781623 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="util" Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781641 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="util" Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781664 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="pull" Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781672 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="pull" Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781680 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="extract" Jul 23 00:23:40.781713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781688 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="extract" Jul 23 00:23:40.782055 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.781773 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="c25d1e91-b3d4-46d4-97a0-8a8454f1cb55" containerName="extract" Jul 23 00:23:40.783819 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.783798 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.786937 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.786910 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 23 00:23:40.786937 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.786923 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 23 00:23:40.787085 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.787005 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l7dvf\"" Jul 23 00:23:40.796023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.796006 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4"] Jul 23 00:23:40.828607 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.828582 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrwp5\" (UniqueName: \"kubernetes.io/projected/150f3485-3e71-49cc-bb75-6550b39d92b7-kube-api-access-wrwp5\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.828683 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.828627 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150f3485-3e71-49cc-bb75-6550b39d92b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.929836 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.929808 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrwp5\" (UniqueName: \"kubernetes.io/projected/150f3485-3e71-49cc-bb75-6550b39d92b7-kube-api-access-wrwp5\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.929919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.929874 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150f3485-3e71-49cc-bb75-6550b39d92b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.930167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.930152 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/150f3485-3e71-49cc-bb75-6550b39d92b7-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:40.938199 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:40.938176 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrwp5\" (UniqueName: \"kubernetes.io/projected/150f3485-3e71-49cc-bb75-6550b39d92b7-kube-api-access-wrwp5\") pod \"cert-manager-operator-controller-manager-55c444bf86-g9hx4\" (UID: \"150f3485-3e71-49cc-bb75-6550b39d92b7\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:41.092304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:41.092271 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" Jul 23 00:23:41.224388 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:41.224359 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4"] Jul 23 00:23:41.227604 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:23:41.227580 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod150f3485_3e71_49cc_bb75_6550b39d92b7.slice/crio-31f035e47b3d46b8939959a6009648ba1c1fc17561755db18b15bdcd24eae481 WatchSource:0}: Error finding container 31f035e47b3d46b8939959a6009648ba1c1fc17561755db18b15bdcd24eae481: Status 404 returned error can't find the container with id 31f035e47b3d46b8939959a6009648ba1c1fc17561755db18b15bdcd24eae481 Jul 23 00:23:42.133561 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:42.133521 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" event={"ID":"150f3485-3e71-49cc-bb75-6550b39d92b7","Type":"ContainerStarted","Data":"31f035e47b3d46b8939959a6009648ba1c1fc17561755db18b15bdcd24eae481"} Jul 23 00:23:43.139117 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:43.139041 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" event={"ID":"150f3485-3e71-49cc-bb75-6550b39d92b7","Type":"ContainerStarted","Data":"96f5d53d45d007427ac2430ef6968005bce90e0afa52191109d3d3355991b654"} Jul 23 00:23:43.163167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:43.163123 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-g9hx4" podStartSLOduration=1.584002043 podStartE2EDuration="3.163108162s" totalImagesPullingTime="1.579106119s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:43.161160945 +0000 UTC m=+214.467308957" watchObservedRunningTime="2026-07-23 00:23:43.163108162 +0000 UTC m=+214.469256155" Jul 23 00:23:44.456169 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.456140 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94"] Jul 23 00:23:44.458527 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.458510 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.461257 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.461234 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2d2n8\"" Jul 23 00:23:44.461376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.461275 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 00:23:44.461376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.461279 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 00:23:44.469286 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.469264 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94"] Jul 23 00:23:44.555644 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.555612 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.555809 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.555648 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzqm9\" (UniqueName: \"kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.555809 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.555741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.656281 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.656251 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.656281 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.656284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzqm9\" (UniqueName: \"kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.656484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.656320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.656603 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.656582 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.656663 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.656618 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.672327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.672296 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzqm9\" (UniqueName: \"kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.766955 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.766897 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:44.893574 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:44.893552 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94"] Jul 23 00:23:44.895982 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:23:44.895956 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fc1c088_cef4_43b1_afb5_2b40253666e1.slice/crio-ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf WatchSource:0}: Error finding container ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf: Status 404 returned error can't find the container with id ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf Jul 23 00:23:45.154594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:45.154565 2575 generic.go:356] "Generic (PLEG): container finished" podID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerID="cfbc025e650242981859af1c50014558f3ed5c5cf2f4b428876959663ceb85a4" exitCode=0 Jul 23 00:23:45.154725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:45.154599 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" event={"ID":"2fc1c088-cef4-43b1-afb5-2b40253666e1","Type":"ContainerDied","Data":"cfbc025e650242981859af1c50014558f3ed5c5cf2f4b428876959663ceb85a4"} Jul 23 00:23:45.154725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:45.154654 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" event={"ID":"2fc1c088-cef4-43b1-afb5-2b40253666e1","Type":"ContainerStarted","Data":"ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf"} Jul 23 00:23:47.295807 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.295781 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v"] Jul 23 00:23:47.297990 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.297976 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.300523 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.300505 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 23 00:23:47.300600 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.300505 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 23 00:23:47.301524 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.301504 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-z9jr5\"" Jul 23 00:23:47.305796 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.305776 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v"] Jul 23 00:23:47.376848 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.376804 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.376958 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.376899 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xgxl\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-kube-api-access-4xgxl\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.478112 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.478048 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.478112 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.478101 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xgxl\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-kube-api-access-4xgxl\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.485944 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.485923 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.486126 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.486109 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xgxl\" (UniqueName: \"kubernetes.io/projected/749d474e-07b8-4337-aea0-d20f02caf9cb-kube-api-access-4xgxl\") pod \"cert-manager-cainjector-559cdd8c87-8zm5v\" (UID: \"749d474e-07b8-4337-aea0-d20f02caf9cb\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.636080 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.636038 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" Jul 23 00:23:47.756098 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:47.755998 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v"] Jul 23 00:23:47.758500 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:23:47.758475 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod749d474e_07b8_4337_aea0_d20f02caf9cb.slice/crio-2282d3aa99593f711b7ec911edea040362eced30dab1d986607bcc53ce4a8889 WatchSource:0}: Error finding container 2282d3aa99593f711b7ec911edea040362eced30dab1d986607bcc53ce4a8889: Status 404 returned error can't find the container with id 2282d3aa99593f711b7ec911edea040362eced30dab1d986607bcc53ce4a8889 Jul 23 00:23:48.169551 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:48.169516 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" event={"ID":"749d474e-07b8-4337-aea0-d20f02caf9cb","Type":"ContainerStarted","Data":"2282d3aa99593f711b7ec911edea040362eced30dab1d986607bcc53ce4a8889"} Jul 23 00:23:48.171574 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:48.171543 2575 generic.go:356] "Generic (PLEG): container finished" podID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerID="966e3a3488173ac75d062e8bcb4f9da76721548f89fbeea67479a186ac7747ed" exitCode=0 Jul 23 00:23:48.171728 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:48.171624 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" event={"ID":"2fc1c088-cef4-43b1-afb5-2b40253666e1","Type":"ContainerDied","Data":"966e3a3488173ac75d062e8bcb4f9da76721548f89fbeea67479a186ac7747ed"} Jul 23 00:23:49.178926 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:49.178892 2575 generic.go:356] "Generic (PLEG): container finished" podID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerID="a264fd27b8036942710b9821d46169d6c151806440c21f256038a75d7ceff295" exitCode=0 Jul 23 00:23:49.178926 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:49.178937 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" event={"ID":"2fc1c088-cef4-43b1-afb5-2b40253666e1","Type":"ContainerDied","Data":"a264fd27b8036942710b9821d46169d6c151806440c21f256038a75d7ceff295"} Jul 23 00:23:50.184611 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.184574 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" event={"ID":"749d474e-07b8-4337-aea0-d20f02caf9cb","Type":"ContainerStarted","Data":"2e9a6718375581ee3c121cec4847fb482033f80a49bdd261159b3d03965b540e"} Jul 23 00:23:50.209478 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.209430 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-8zm5v" podStartSLOduration=0.909640473 podStartE2EDuration="3.209411811s" totalImagesPullingTime="2.299771338s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:50.207987946 +0000 UTC m=+221.514135944" watchObservedRunningTime="2026-07-23 00:23:50.209411811 +0000 UTC m=+221.515559806" Jul 23 00:23:50.319442 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.319385 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:50.406545 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.406515 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util\") pod \"2fc1c088-cef4-43b1-afb5-2b40253666e1\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " Jul 23 00:23:50.406702 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.406573 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bzqm9\" (UniqueName: \"kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9\") pod \"2fc1c088-cef4-43b1-afb5-2b40253666e1\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " Jul 23 00:23:50.406702 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.406625 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle\") pod \"2fc1c088-cef4-43b1-afb5-2b40253666e1\" (UID: \"2fc1c088-cef4-43b1-afb5-2b40253666e1\") " Jul 23 00:23:50.407007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.406984 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle" (OuterVolumeSpecName: "bundle") pod "2fc1c088-cef4-43b1-afb5-2b40253666e1" (UID: "2fc1c088-cef4-43b1-afb5-2b40253666e1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:23:50.408612 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.408591 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9" (OuterVolumeSpecName: "kube-api-access-bzqm9") pod "2fc1c088-cef4-43b1-afb5-2b40253666e1" (UID: "2fc1c088-cef4-43b1-afb5-2b40253666e1"). InnerVolumeSpecName "kube-api-access-bzqm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:23:50.412923 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.412883 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util" (OuterVolumeSpecName: "util") pod "2fc1c088-cef4-43b1-afb5-2b40253666e1" (UID: "2fc1c088-cef4-43b1-afb5-2b40253666e1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:23:50.507695 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.507666 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bzqm9\" (UniqueName: \"kubernetes.io/projected/2fc1c088-cef4-43b1-afb5-2b40253666e1-kube-api-access-bzqm9\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:50.507695 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.507691 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:50.507695 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:50.507700 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/2fc1c088-cef4-43b1-afb5-2b40253666e1-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:23:51.190564 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:51.190524 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" event={"ID":"2fc1c088-cef4-43b1-afb5-2b40253666e1","Type":"ContainerDied","Data":"ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf"} Jul 23 00:23:51.190564 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:51.190550 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f9fr94" Jul 23 00:23:51.190948 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:51.190583 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ed2fe53bd0aa3b3108f3fa5796c655c10938b8cbecc9bf454417e657c96a13bf" Jul 23 00:23:56.871366 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871328 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2dhg"] Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871624 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="util" Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871635 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="util" Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871648 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="extract" Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871654 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="extract" Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871668 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="pull" Jul 23 00:23:56.871718 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871674 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="pull" Jul 23 00:23:56.871966 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.871726 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fc1c088-cef4-43b1-afb5-2b40253666e1" containerName="extract" Jul 23 00:23:56.875610 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.875594 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:56.878823 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.878804 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-db7wh\"" Jul 23 00:23:56.883794 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.883770 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2dhg"] Jul 23 00:23:56.955168 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.955139 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:56.955272 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:56.955190 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5vss\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-kube-api-access-t5vss\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.056213 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.056179 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5vss\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-kube-api-access-t5vss\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.056321 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.056236 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.063705 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.063679 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-bound-sa-token\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.063845 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.063787 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5vss\" (UniqueName: \"kubernetes.io/projected/6d1fddb3-c6d5-4fe4-b97a-6b1442239409-kube-api-access-t5vss\") pod \"cert-manager-55b4d954d5-z2dhg\" (UID: \"6d1fddb3-c6d5-4fe4-b97a-6b1442239409\") " pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.184569 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.184492 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-z2dhg" Jul 23 00:23:57.310128 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:57.310102 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-z2dhg"] Jul 23 00:23:57.312218 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:23:57.312194 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6d1fddb3_c6d5_4fe4_b97a_6b1442239409.slice/crio-f4930156034e790d44f286c5246e026ffe40a2c2452b8c22d559612d3d424d01 WatchSource:0}: Error finding container f4930156034e790d44f286c5246e026ffe40a2c2452b8c22d559612d3d424d01: Status 404 returned error can't find the container with id f4930156034e790d44f286c5246e026ffe40a2c2452b8c22d559612d3d424d01 Jul 23 00:23:58.220985 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:58.220951 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z2dhg" event={"ID":"6d1fddb3-c6d5-4fe4-b97a-6b1442239409","Type":"ContainerStarted","Data":"357565e1a87a2c157317b3706578822f45b664f2fe331f206cf8de707234b4e4"} Jul 23 00:23:58.221308 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:58.220992 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-z2dhg" event={"ID":"6d1fddb3-c6d5-4fe4-b97a-6b1442239409","Type":"ContainerStarted","Data":"f4930156034e790d44f286c5246e026ffe40a2c2452b8c22d559612d3d424d01"} Jul 23 00:23:58.241137 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:23:58.241092 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-z2dhg" podStartSLOduration=2.241075178 podStartE2EDuration="2.241075178s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:23:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:23:58.240893944 +0000 UTC m=+229.547041938" watchObservedRunningTime="2026-07-23 00:23:58.241075178 +0000 UTC m=+229.547223174" Jul 23 00:24:05.968696 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.968662 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v"] Jul 23 00:24:05.971010 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.970993 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:05.973474 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.973449 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 23 00:24:05.974646 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.974628 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 23 00:24:05.974745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.974633 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-2d2n8\"" Jul 23 00:24:05.980973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:05.980952 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v"] Jul 23 00:24:06.021985 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.021961 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.022079 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.022007 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z5cx8\" (UniqueName: \"kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.022121 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.022072 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.123246 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.123210 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.123341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.123260 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.123341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.123295 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z5cx8\" (UniqueName: \"kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.123608 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.123589 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.123608 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.123600 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.131860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.131818 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z5cx8\" (UniqueName: \"kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.281117 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.281056 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:06.405863 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:06.405819 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v"] Jul 23 00:24:06.407847 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:24:06.407799 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9bc0603d_43b4_4bde_bad5_79cac44bc4b4.slice/crio-cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223 WatchSource:0}: Error finding container cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223: Status 404 returned error can't find the container with id cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223 Jul 23 00:24:07.260924 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:07.260886 2575 generic.go:356] "Generic (PLEG): container finished" podID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerID="c450c464862057639606a8d76d85aab1ece47c00b1dd051399cac762deb99c9b" exitCode=0 Jul 23 00:24:07.261290 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:07.260968 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" event={"ID":"9bc0603d-43b4-4bde-bad5-79cac44bc4b4","Type":"ContainerDied","Data":"c450c464862057639606a8d76d85aab1ece47c00b1dd051399cac762deb99c9b"} Jul 23 00:24:07.261290 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:07.261016 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" event={"ID":"9bc0603d-43b4-4bde-bad5-79cac44bc4b4","Type":"ContainerStarted","Data":"cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223"} Jul 23 00:24:08.266807 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:08.266722 2575 generic.go:356] "Generic (PLEG): container finished" podID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerID="630b68b2c4c6917bcff9d2d7d2bb35571b485301fd5c8e6d8b2d45b627e11cf0" exitCode=0 Jul 23 00:24:08.266807 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:08.266781 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" event={"ID":"9bc0603d-43b4-4bde-bad5-79cac44bc4b4","Type":"ContainerDied","Data":"630b68b2c4c6917bcff9d2d7d2bb35571b485301fd5c8e6d8b2d45b627e11cf0"} Jul 23 00:24:09.272390 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:09.272351 2575 generic.go:356] "Generic (PLEG): container finished" podID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerID="819f4ed9650cdf7faf8677752229a077265a69ddb9ec98fc1a4eed4f3e78aafa" exitCode=0 Jul 23 00:24:09.272741 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:09.272430 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" event={"ID":"9bc0603d-43b4-4bde-bad5-79cac44bc4b4","Type":"ContainerDied","Data":"819f4ed9650cdf7faf8677752229a077265a69ddb9ec98fc1a4eed4f3e78aafa"} Jul 23 00:24:10.396141 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.396111 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:10.458893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.458861 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util\") pod \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " Jul 23 00:24:10.459026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.458946 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle\") pod \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " Jul 23 00:24:10.459026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.458988 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-z5cx8\" (UniqueName: \"kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8\") pod \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\" (UID: \"9bc0603d-43b4-4bde-bad5-79cac44bc4b4\") " Jul 23 00:24:10.459631 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.459584 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle" (OuterVolumeSpecName: "bundle") pod "9bc0603d-43b4-4bde-bad5-79cac44bc4b4" (UID: "9bc0603d-43b4-4bde-bad5-79cac44bc4b4"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:24:10.460946 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.460922 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8" (OuterVolumeSpecName: "kube-api-access-z5cx8") pod "9bc0603d-43b4-4bde-bad5-79cac44bc4b4" (UID: "9bc0603d-43b4-4bde-bad5-79cac44bc4b4"). InnerVolumeSpecName "kube-api-access-z5cx8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:24:10.464438 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.464418 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util" (OuterVolumeSpecName: "util") pod "9bc0603d-43b4-4bde-bad5-79cac44bc4b4" (UID: "9bc0603d-43b4-4bde-bad5-79cac44bc4b4"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:24:10.560276 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.560245 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:24:10.560276 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.560272 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-z5cx8\" (UniqueName: \"kubernetes.io/projected/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-kube-api-access-z5cx8\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:24:10.560468 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:10.560284 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/9bc0603d-43b4-4bde-bad5-79cac44bc4b4-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:24:11.283268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:11.283245 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" Jul 23 00:24:11.283415 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:11.283234 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796af77l7v" event={"ID":"9bc0603d-43b4-4bde-bad5-79cac44bc4b4","Type":"ContainerDied","Data":"cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223"} Jul 23 00:24:11.283415 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:11.283373 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cba2a009ff23f9b5f2cb415ddeabd771854a51a0e90a029a12f031ba0d819223" Jul 23 00:24:21.495599 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495566 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w"] Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495862 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="pull" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495873 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="pull" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495890 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="extract" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495895 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="extract" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495904 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="util" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495910 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="util" Jul 23 00:24:21.495970 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.495961 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="9bc0603d-43b4-4bde-bad5-79cac44bc4b4" containerName="extract" Jul 23 00:24:21.498368 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.498352 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.501985 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.501958 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 23 00:24:21.503342 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.503313 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 23 00:24:21.503459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.503355 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-l4272\"" Jul 23 00:24:21.503459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.503405 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 23 00:24:21.503459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.503420 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 23 00:24:21.503635 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.503412 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 23 00:24:21.513233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.513202 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w"] Jul 23 00:24:21.646525 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.646497 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-metrics-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.646666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.646540 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvp9p\" (UniqueName: \"kubernetes.io/projected/e42549f7-001a-449a-9d05-630286b3ba10-kube-api-access-vvp9p\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.646666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.646570 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.646666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.646616 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e42549f7-001a-449a-9d05-630286b3ba10-manager-config\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.747859 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.747780 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.747859 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.747813 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e42549f7-001a-449a-9d05-630286b3ba10-manager-config\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.748055 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.747879 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-metrics-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.748055 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.747914 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vvp9p\" (UniqueName: \"kubernetes.io/projected/e42549f7-001a-449a-9d05-630286b3ba10-kube-api-access-vvp9p\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.748637 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.748611 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e42549f7-001a-449a-9d05-630286b3ba10-manager-config\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.750263 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.750238 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-metrics-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.750349 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.750284 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e42549f7-001a-449a-9d05-630286b3ba10-cert\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.763268 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.763245 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvp9p\" (UniqueName: \"kubernetes.io/projected/e42549f7-001a-449a-9d05-630286b3ba10-kube-api-access-vvp9p\") pod \"lws-controller-manager-6958b76574-rwk2w\" (UID: \"e42549f7-001a-449a-9d05-630286b3ba10\") " pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.806986 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.806966 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:21.945628 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:21.945598 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w"] Jul 23 00:24:21.948960 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:24:21.948933 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode42549f7_001a_449a_9d05_630286b3ba10.slice/crio-36cfb8571dd6880c0b212e6a75d6c78693423117be23d9cb76ed76db1ce20489 WatchSource:0}: Error finding container 36cfb8571dd6880c0b212e6a75d6c78693423117be23d9cb76ed76db1ce20489: Status 404 returned error can't find the container with id 36cfb8571dd6880c0b212e6a75d6c78693423117be23d9cb76ed76db1ce20489 Jul 23 00:24:22.159002 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.158970 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk"] Jul 23 00:24:22.161895 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.161879 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.164651 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.164631 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 23 00:24:22.164767 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.164732 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 23 00:24:22.164893 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.164879 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 23 00:24:22.165061 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.165047 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 23 00:24:22.165136 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.165082 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-rqkdt\"" Jul 23 00:24:22.179012 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.178987 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk"] Jul 23 00:24:22.251879 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.251854 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-webhook-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.251981 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.251882 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-manifests\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.251981 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.251901 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.251981 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.251952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2ccp\" (UniqueName: \"kubernetes.io/projected/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-kube-api-access-q2ccp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.252084 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.251995 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-tmp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.327988 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.327961 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" event={"ID":"e42549f7-001a-449a-9d05-630286b3ba10","Type":"ContainerStarted","Data":"36cfb8571dd6880c0b212e6a75d6c78693423117be23d9cb76ed76db1ce20489"} Jul 23 00:24:22.353326 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353304 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2ccp\" (UniqueName: \"kubernetes.io/projected/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-kube-api-access-q2ccp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.353392 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353335 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-tmp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.353392 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353374 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-webhook-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.353462 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353397 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-manifests\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.353582 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353563 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.353859 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.353821 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-manifests\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.355564 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.355541 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-tmp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.355769 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.355754 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-webhook-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.355810 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.355766 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.361603 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.361579 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2ccp\" (UniqueName: \"kubernetes.io/projected/8394b76f-9271-4d86-b2dc-ec9d08b6ab26-kube-api-access-q2ccp\") pod \"opendatahub-operator-controller-manager-8844c5db8-d6tdk\" (UID: \"8394b76f-9271-4d86-b2dc-ec9d08b6ab26\") " pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.472958 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.472882 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:22.614311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:22.614286 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk"] Jul 23 00:24:22.616414 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:24:22.616389 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8394b76f_9271_4d86_b2dc_ec9d08b6ab26.slice/crio-feab64629cdd7d091c9ed5892e1d480ed32a5c059cb590dc20a3d20761ff0ea7 WatchSource:0}: Error finding container feab64629cdd7d091c9ed5892e1d480ed32a5c059cb590dc20a3d20761ff0ea7: Status 404 returned error can't find the container with id feab64629cdd7d091c9ed5892e1d480ed32a5c059cb590dc20a3d20761ff0ea7 Jul 23 00:24:23.334056 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:23.334019 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" event={"ID":"8394b76f-9271-4d86-b2dc-ec9d08b6ab26","Type":"ContainerStarted","Data":"feab64629cdd7d091c9ed5892e1d480ed32a5c059cb590dc20a3d20761ff0ea7"} Jul 23 00:24:24.342306 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:24.342267 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" event={"ID":"e42549f7-001a-449a-9d05-630286b3ba10","Type":"ContainerStarted","Data":"8119a0eb9e76889317904bff0aedf776b6c29c7abd0480d374bb8c16092facbd"} Jul 23 00:24:24.342748 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:24.342363 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:24.359282 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:24.359198 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" podStartSLOduration=1.379507937 podStartE2EDuration="3.359177761s" totalImagesPullingTime="1.979669824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:24:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:24.358154192 +0000 UTC m=+255.664302188" watchObservedRunningTime="2026-07-23 00:24:24.359177761 +0000 UTC m=+255.665325964" Jul 23 00:24:26.353199 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:26.353165 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" event={"ID":"8394b76f-9271-4d86-b2dc-ec9d08b6ab26","Type":"ContainerStarted","Data":"50648546416a5a1ad0349116f5c4a4997fb51371323e108c99095d3dc7f91d76"} Jul 23 00:24:27.358577 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:27.358544 2575 generic.go:356] "Generic (PLEG): container finished" podID="8394b76f-9271-4d86-b2dc-ec9d08b6ab26" containerID="50648546416a5a1ad0349116f5c4a4997fb51371323e108c99095d3dc7f91d76" exitCode=0 Jul 23 00:24:27.358969 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:27.358632 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" event={"ID":"8394b76f-9271-4d86-b2dc-ec9d08b6ab26","Type":"ContainerDied","Data":"50648546416a5a1ad0349116f5c4a4997fb51371323e108c99095d3dc7f91d76"} Jul 23 00:24:28.365594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:28.365554 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" event={"ID":"8394b76f-9271-4d86-b2dc-ec9d08b6ab26","Type":"ContainerStarted","Data":"92e80a676712dfb607eedff09b23daa39e41642aab03bc63290cbfdf365a2462"} Jul 23 00:24:28.366048 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:28.365660 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:28.388563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:28.388513 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" podStartSLOduration=1.7994698009999999 podStartE2EDuration="6.388500694s" totalImagesPullingTime="3.595078271s" totalInitContainerRuntime="993.952622ms" isStatefulPod=false podCreationTimestamp="2026-07-23 00:24:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:28.386104127 +0000 UTC m=+259.692252125" watchObservedRunningTime="2026-07-23 00:24:28.388500694 +0000 UTC m=+259.694648688" Jul 23 00:24:35.350341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:35.350262 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-6958b76574-rwk2w" Jul 23 00:24:39.372297 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:39.372265 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8844c5db8-d6tdk" Jul 23 00:24:48.639522 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.639491 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k"] Jul 23 00:24:48.649361 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.649335 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.651960 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.651936 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 23 00:24:48.652107 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.652002 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 23 00:24:48.652107 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.652002 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-hxtfh\"" Jul 23 00:24:48.652285 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.652265 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 23 00:24:48.652355 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.652315 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 23 00:24:48.652355 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.652337 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 23 00:24:48.653070 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.653050 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 23 00:24:48.662190 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.662165 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k"] Jul 23 00:24:48.668874 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.668850 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tdt7\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-kube-api-access-2tdt7\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.668973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.668891 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.668973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.668964 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.669058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.668996 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.669058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.669021 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.669135 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.669055 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.669135 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.669081 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.769902 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.769864 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.769934 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.769965 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.770010 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tdt7\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-kube-api-access-2tdt7\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.770051 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.770115 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.770152 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.770959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.770928 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.772556 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.772528 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.772685 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.772592 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.772685 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.772603 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.772685 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.772665 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.777553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.777524 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.777649 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.777576 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tdt7\" (UniqueName: \"kubernetes.io/projected/6a58d0bb-9fa6-491c-a88a-8f980619d0b1-kube-api-access-2tdt7\") pod \"istiod-openshift-gateway-94bb8fbfd-qcg8k\" (UID: \"6a58d0bb-9fa6-491c-a88a-8f980619d0b1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:48.960322 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:48.960250 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:49.091625 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:49.091597 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k"] Jul 23 00:24:49.094016 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:24:49.093978 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a58d0bb_9fa6_491c_a88a_8f980619d0b1.slice/crio-5f5f5352d74e336a2ccd1eb953f62a2400fd89186d45785b57922ec3d5cb82b0 WatchSource:0}: Error finding container 5f5f5352d74e336a2ccd1eb953f62a2400fd89186d45785b57922ec3d5cb82b0: Status 404 returned error can't find the container with id 5f5f5352d74e336a2ccd1eb953f62a2400fd89186d45785b57922ec3d5cb82b0 Jul 23 00:24:49.473235 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:49.473201 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" event={"ID":"6a58d0bb-9fa6-491c-a88a-8f980619d0b1","Type":"ContainerStarted","Data":"5f5f5352d74e336a2ccd1eb953f62a2400fd89186d45785b57922ec3d5cb82b0"} Jul 23 00:24:51.407322 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:51.407264 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 00:24:51.407618 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:51.407349 2575 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 23 00:24:51.485054 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:51.485024 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" event={"ID":"6a58d0bb-9fa6-491c-a88a-8f980619d0b1","Type":"ContainerStarted","Data":"ba641ef122544b3924fe6aa7e0e880e78fb06866f9197a15e2779a77d7769e28"} Jul 23 00:24:51.485192 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:51.485174 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:24:51.521114 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:51.521061 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" podStartSLOduration=1.210343687 podStartE2EDuration="3.521043515s" totalImagesPullingTime="2.310699828s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:24:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:24:51.517345074 +0000 UTC m=+282.823493071" watchObservedRunningTime="2026-07-23 00:24:51.521043515 +0000 UTC m=+282.827191508" Jul 23 00:24:52.491681 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:52.491648 2575 patch_prober.go:28] interesting pod/istiod-openshift-gateway-94bb8fbfd-qcg8k container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 23 00:24:52.492129 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:52.491712 2575 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" podUID="6a58d0bb-9fa6-491c-a88a-8f980619d0b1" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 23 00:24:55.491272 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:24:55.491240 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-qcg8k" Jul 23 00:25:09.072555 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:09.072524 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:25:09.075697 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:09.075664 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:25:09.078559 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:09.078542 2575 kubelet.go:1648] "Image garbage collection succeeded" Jul 23 00:25:58.202880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.202845 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg"] Jul 23 00:25:58.206244 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.206227 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.209141 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.209117 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 00:25:58.210449 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.210432 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6blfh\"" Jul 23 00:25:58.210515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.210446 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 00:25:58.214709 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.214678 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.214811 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.214738 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.214896 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.214808 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8czlz\" (UniqueName: \"kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.215553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.215525 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg"] Jul 23 00:25:58.316206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.316175 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.316328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.316217 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.316328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.316256 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8czlz\" (UniqueName: \"kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.316565 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.316538 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.316638 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.316596 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.324732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.324700 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8czlz\" (UniqueName: \"kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.516086 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.516001 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:25:58.648317 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.648291 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg"] Jul 23 00:25:58.650122 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:25:58.650089 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e53bc96_13d5_4716_989e_c2998b8042fd.slice/crio-1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a WatchSource:0}: Error finding container 1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a: Status 404 returned error can't find the container with id 1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a Jul 23 00:25:58.782207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.782148 2575 generic.go:356] "Generic (PLEG): container finished" podID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerID="70c19d00088a609621d2c0b02fc92d9d53865f9bfece76971b5fa93de990348c" exitCode=0 Jul 23 00:25:58.782317 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.782237 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" event={"ID":"0e53bc96-13d5-4716-989e-c2998b8042fd","Type":"ContainerDied","Data":"70c19d00088a609621d2c0b02fc92d9d53865f9bfece76971b5fa93de990348c"} Jul 23 00:25:58.782317 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.782298 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" event={"ID":"0e53bc96-13d5-4716-989e-c2998b8042fd","Type":"ContainerStarted","Data":"1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a"} Jul 23 00:25:58.782633 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.782619 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:25:58.806668 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.806644 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2"] Jul 23 00:25:58.809963 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.809947 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.821289 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.821268 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.821387 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.821321 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.821387 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.821348 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djfvq\" (UniqueName: \"kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.826427 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.826408 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2"] Jul 23 00:25:58.922262 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.922237 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.922363 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.922269 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djfvq\" (UniqueName: \"kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.922363 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.922320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.922579 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.922560 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.922656 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.922588 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:58.931485 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:58.931463 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djfvq\" (UniqueName: \"kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:59.131936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.131908 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:25:59.207722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.207692 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd"] Jul 23 00:25:59.211519 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.211498 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.224573 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.224542 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.224706 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.224660 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcmts\" (UniqueName: \"kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.224769 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.224725 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.225535 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.225514 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd"] Jul 23 00:25:59.268655 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.268635 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2"] Jul 23 00:25:59.270799 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:25:59.270776 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51372e8b_ba0d_41f1_9535_706617fe2a3e.slice/crio-bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564 WatchSource:0}: Error finding container bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564: Status 404 returned error can't find the container with id bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564 Jul 23 00:25:59.326062 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.326037 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcmts\" (UniqueName: \"kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.326153 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.326102 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.326153 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.326122 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.326416 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.326396 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.326495 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.326437 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.335897 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.335875 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcmts\" (UniqueName: \"kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.521557 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.521534 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:25:59.600578 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.600549 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf"] Jul 23 00:25:59.604605 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.604585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.615725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.615694 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf"] Jul 23 00:25:59.627760 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.627733 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9gbz\" (UniqueName: \"kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.627898 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.627778 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.627898 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.627796 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.653949 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.653930 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd"] Jul 23 00:25:59.655933 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:25:59.655911 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35c022c0_1485_4a3e_ae0b_964f4b17a244.slice/crio-21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28 WatchSource:0}: Error finding container 21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28: Status 404 returned error can't find the container with id 21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28 Jul 23 00:25:59.728661 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.728632 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9gbz\" (UniqueName: \"kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.728756 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.728675 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.728756 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.728693 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.729075 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.729056 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.729133 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.729074 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.738200 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.738157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9gbz\" (UniqueName: \"kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:25:59.788669 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.788640 2575 generic.go:356] "Generic (PLEG): container finished" podID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerID="88a50cd0b043df086e46032b42a967681bac0b7ef66524dbb8a61e99ee5e5e61" exitCode=0 Jul 23 00:25:59.788758 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.788723 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" event={"ID":"0e53bc96-13d5-4716-989e-c2998b8042fd","Type":"ContainerDied","Data":"88a50cd0b043df086e46032b42a967681bac0b7ef66524dbb8a61e99ee5e5e61"} Jul 23 00:25:59.790310 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.790288 2575 generic.go:356] "Generic (PLEG): container finished" podID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerID="f180efa9dc163df73e4ba6647288e0ec8a65b60692c0b14772811c7c7791ec26" exitCode=0 Jul 23 00:25:59.790394 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.790365 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" event={"ID":"35c022c0-1485-4a3e-ae0b-964f4b17a244","Type":"ContainerDied","Data":"f180efa9dc163df73e4ba6647288e0ec8a65b60692c0b14772811c7c7791ec26"} Jul 23 00:25:59.790447 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.790411 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" event={"ID":"35c022c0-1485-4a3e-ae0b-964f4b17a244","Type":"ContainerStarted","Data":"21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28"} Jul 23 00:25:59.792008 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.791987 2575 generic.go:356] "Generic (PLEG): container finished" podID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerID="118889ef95cf96090603b848f88670b4eee92ed0eaa98545e697974585aaaca3" exitCode=0 Jul 23 00:25:59.792104 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.792047 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" event={"ID":"51372e8b-ba0d-41f1-9535-706617fe2a3e","Type":"ContainerDied","Data":"118889ef95cf96090603b848f88670b4eee92ed0eaa98545e697974585aaaca3"} Jul 23 00:25:59.792104 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.792084 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" event={"ID":"51372e8b-ba0d-41f1-9535-706617fe2a3e","Type":"ContainerStarted","Data":"bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564"} Jul 23 00:25:59.946296 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:25:59.946266 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:26:00.070890 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.070860 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf"] Jul 23 00:26:00.073089 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:26:00.073061 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd18608ad_add2_4401_aceb_e7cecde3ea08.slice/crio-9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca WatchSource:0}: Error finding container 9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca: Status 404 returned error can't find the container with id 9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca Jul 23 00:26:00.798685 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.798656 2575 generic.go:356] "Generic (PLEG): container finished" podID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerID="7d2793f1e8783a4170be88bcb691342adab48e6a7f06927051a5c9cb3d1c488b" exitCode=0 Jul 23 00:26:00.799044 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.798702 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" event={"ID":"0e53bc96-13d5-4716-989e-c2998b8042fd","Type":"ContainerDied","Data":"7d2793f1e8783a4170be88bcb691342adab48e6a7f06927051a5c9cb3d1c488b"} Jul 23 00:26:00.800437 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.800419 2575 generic.go:356] "Generic (PLEG): container finished" podID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerID="893d9b0457dbb8f66deaaf2f2e4e00fb50622207980d57aa94b1f552ec4c3a65" exitCode=0 Jul 23 00:26:00.800539 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.800480 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" event={"ID":"35c022c0-1485-4a3e-ae0b-964f4b17a244","Type":"ContainerDied","Data":"893d9b0457dbb8f66deaaf2f2e4e00fb50622207980d57aa94b1f552ec4c3a65"} Jul 23 00:26:00.801936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.801914 2575 generic.go:356] "Generic (PLEG): container finished" podID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerID="47e944c849c3e47607654b3887005a6ea556625eb43fef807927e7e4623b2027" exitCode=0 Jul 23 00:26:00.802020 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.802001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" event={"ID":"d18608ad-add2-4401-aceb-e7cecde3ea08","Type":"ContainerDied","Data":"47e944c849c3e47607654b3887005a6ea556625eb43fef807927e7e4623b2027"} Jul 23 00:26:00.802084 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.802042 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" event={"ID":"d18608ad-add2-4401-aceb-e7cecde3ea08","Type":"ContainerStarted","Data":"9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca"} Jul 23 00:26:00.804082 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.804060 2575 generic.go:356] "Generic (PLEG): container finished" podID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerID="74dbe08a6b870f6c2821488b1b7d99f87e242ec523b79aa786c4c8cbf0beacdd" exitCode=0 Jul 23 00:26:00.804136 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:00.804095 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" event={"ID":"51372e8b-ba0d-41f1-9535-706617fe2a3e","Type":"ContainerDied","Data":"74dbe08a6b870f6c2821488b1b7d99f87e242ec523b79aa786c4c8cbf0beacdd"} Jul 23 00:26:01.810736 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:01.810700 2575 generic.go:356] "Generic (PLEG): container finished" podID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerID="d37f759eff6f027fc627fc8b56be08de47770da0863b46b4e92514677ca9c611" exitCode=0 Jul 23 00:26:01.811145 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:01.810816 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" event={"ID":"35c022c0-1485-4a3e-ae0b-964f4b17a244","Type":"ContainerDied","Data":"d37f759eff6f027fc627fc8b56be08de47770da0863b46b4e92514677ca9c611"} Jul 23 00:26:01.812689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:01.812670 2575 generic.go:356] "Generic (PLEG): container finished" podID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerID="7edbe6e6ee4407a105afef9e4e0ce216d61292fec5fd4ee9cb10b1eb31f8e52c" exitCode=0 Jul 23 00:26:01.812784 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:01.812750 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" event={"ID":"51372e8b-ba0d-41f1-9535-706617fe2a3e","Type":"ContainerDied","Data":"7edbe6e6ee4407a105afef9e4e0ce216d61292fec5fd4ee9cb10b1eb31f8e52c"} Jul 23 00:26:01.976459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:01.976432 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:26:02.047509 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.047481 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle\") pod \"0e53bc96-13d5-4716-989e-c2998b8042fd\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " Jul 23 00:26:02.047638 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.047533 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util\") pod \"0e53bc96-13d5-4716-989e-c2998b8042fd\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " Jul 23 00:26:02.047638 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.047570 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8czlz\" (UniqueName: \"kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz\") pod \"0e53bc96-13d5-4716-989e-c2998b8042fd\" (UID: \"0e53bc96-13d5-4716-989e-c2998b8042fd\") " Jul 23 00:26:02.048119 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.048096 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle" (OuterVolumeSpecName: "bundle") pod "0e53bc96-13d5-4716-989e-c2998b8042fd" (UID: "0e53bc96-13d5-4716-989e-c2998b8042fd"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:02.049754 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.049723 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz" (OuterVolumeSpecName: "kube-api-access-8czlz") pod "0e53bc96-13d5-4716-989e-c2998b8042fd" (UID: "0e53bc96-13d5-4716-989e-c2998b8042fd"). InnerVolumeSpecName "kube-api-access-8czlz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:26:02.052804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.052771 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util" (OuterVolumeSpecName: "util") pod "0e53bc96-13d5-4716-989e-c2998b8042fd" (UID: "0e53bc96-13d5-4716-989e-c2998b8042fd"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:02.148207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.148184 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:02.148207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.148209 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8czlz\" (UniqueName: \"kubernetes.io/projected/0e53bc96-13d5-4716-989e-c2998b8042fd-kube-api-access-8czlz\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:02.148341 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.148223 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/0e53bc96-13d5-4716-989e-c2998b8042fd-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:02.819338 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.819306 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" Jul 23 00:26:02.819748 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.819307 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg" event={"ID":"0e53bc96-13d5-4716-989e-c2998b8042fd","Type":"ContainerDied","Data":"1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a"} Jul 23 00:26:02.819748 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.819438 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1feb82d0d8d5ee9a9a20a031edcf4bc8fa5b4336020e40cb62544a982229299a" Jul 23 00:26:02.821151 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.821123 2575 generic.go:356] "Generic (PLEG): container finished" podID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerID="b75a6720675b57d2f02fca7320800907462410b866fc6785102699990c5f1941" exitCode=0 Jul 23 00:26:02.821279 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:02.821208 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" event={"ID":"d18608ad-add2-4401-aceb-e7cecde3ea08","Type":"ContainerDied","Data":"b75a6720675b57d2f02fca7320800907462410b866fc6785102699990c5f1941"} Jul 23 00:26:03.020446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.020424 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:26:03.024209 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.024191 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:26:03.055531 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055505 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcmts\" (UniqueName: \"kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts\") pod \"35c022c0-1485-4a3e-ae0b-964f4b17a244\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " Jul 23 00:26:03.055650 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055609 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle\") pod \"35c022c0-1485-4a3e-ae0b-964f4b17a244\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " Jul 23 00:26:03.055714 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055666 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle\") pod \"51372e8b-ba0d-41f1-9535-706617fe2a3e\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " Jul 23 00:26:03.055770 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055747 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util\") pod \"51372e8b-ba0d-41f1-9535-706617fe2a3e\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " Jul 23 00:26:03.055895 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055778 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djfvq\" (UniqueName: \"kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq\") pod \"51372e8b-ba0d-41f1-9535-706617fe2a3e\" (UID: \"51372e8b-ba0d-41f1-9535-706617fe2a3e\") " Jul 23 00:26:03.055895 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.055822 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util\") pod \"35c022c0-1485-4a3e-ae0b-964f4b17a244\" (UID: \"35c022c0-1485-4a3e-ae0b-964f4b17a244\") " Jul 23 00:26:03.056482 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.056443 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle" (OuterVolumeSpecName: "bundle") pod "51372e8b-ba0d-41f1-9535-706617fe2a3e" (UID: "51372e8b-ba0d-41f1-9535-706617fe2a3e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:03.056647 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.056618 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle" (OuterVolumeSpecName: "bundle") pod "35c022c0-1485-4a3e-ae0b-964f4b17a244" (UID: "35c022c0-1485-4a3e-ae0b-964f4b17a244"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:03.058980 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.058956 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts" (OuterVolumeSpecName: "kube-api-access-xcmts") pod "35c022c0-1485-4a3e-ae0b-964f4b17a244" (UID: "35c022c0-1485-4a3e-ae0b-964f4b17a244"). InnerVolumeSpecName "kube-api-access-xcmts". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:26:03.059203 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.059171 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq" (OuterVolumeSpecName: "kube-api-access-djfvq") pod "51372e8b-ba0d-41f1-9535-706617fe2a3e" (UID: "51372e8b-ba0d-41f1-9535-706617fe2a3e"). InnerVolumeSpecName "kube-api-access-djfvq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:26:03.062328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.062307 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util" (OuterVolumeSpecName: "util") pod "35c022c0-1485-4a3e-ae0b-964f4b17a244" (UID: "35c022c0-1485-4a3e-ae0b-964f4b17a244"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:03.063368 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.063347 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util" (OuterVolumeSpecName: "util") pod "51372e8b-ba0d-41f1-9535-706617fe2a3e" (UID: "51372e8b-ba0d-41f1-9535-706617fe2a3e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:03.156901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156820 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.156901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156864 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djfvq\" (UniqueName: \"kubernetes.io/projected/51372e8b-ba0d-41f1-9535-706617fe2a3e-kube-api-access-djfvq\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.156901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156879 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.156901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156892 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcmts\" (UniqueName: \"kubernetes.io/projected/35c022c0-1485-4a3e-ae0b-964f4b17a244-kube-api-access-xcmts\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.157086 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156906 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35c022c0-1485-4a3e-ae0b-964f4b17a244-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.157086 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.156918 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/51372e8b-ba0d-41f1-9535-706617fe2a3e-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:03.828085 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.828051 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" event={"ID":"35c022c0-1485-4a3e-ae0b-964f4b17a244","Type":"ContainerDied","Data":"21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28"} Jul 23 00:26:03.828498 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.828103 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="21d83761bd0e2a0e74e28a4c164c67af91a5ad456bb63d2a8f6babfd2752ef28" Jul 23 00:26:03.828498 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.828061 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd" Jul 23 00:26:03.830312 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.830279 2575 generic.go:356] "Generic (PLEG): container finished" podID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerID="68299f23db00a874ac02188ab6ce46055703084517a332a40609b8ab856d1ae6" exitCode=0 Jul 23 00:26:03.830452 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.830316 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" event={"ID":"d18608ad-add2-4401-aceb-e7cecde3ea08","Type":"ContainerDied","Data":"68299f23db00a874ac02188ab6ce46055703084517a332a40609b8ab856d1ae6"} Jul 23 00:26:03.832255 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.832234 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" event={"ID":"51372e8b-ba0d-41f1-9535-706617fe2a3e","Type":"ContainerDied","Data":"bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564"} Jul 23 00:26:03.832353 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.832271 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bd4a8a02675de4d992f6c9a65ba3ef4071fe23a30117ff92dd4c674aac7ca564" Jul 23 00:26:03.832353 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:03.832238 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2" Jul 23 00:26:04.962419 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:04.962397 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:26:05.073482 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.073453 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle\") pod \"d18608ad-add2-4401-aceb-e7cecde3ea08\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " Jul 23 00:26:05.073621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.073503 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util\") pod \"d18608ad-add2-4401-aceb-e7cecde3ea08\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " Jul 23 00:26:05.073621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.073536 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9gbz\" (UniqueName: \"kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz\") pod \"d18608ad-add2-4401-aceb-e7cecde3ea08\" (UID: \"d18608ad-add2-4401-aceb-e7cecde3ea08\") " Jul 23 00:26:05.073952 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.073916 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle" (OuterVolumeSpecName: "bundle") pod "d18608ad-add2-4401-aceb-e7cecde3ea08" (UID: "d18608ad-add2-4401-aceb-e7cecde3ea08"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:05.075588 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.075565 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz" (OuterVolumeSpecName: "kube-api-access-w9gbz") pod "d18608ad-add2-4401-aceb-e7cecde3ea08" (UID: "d18608ad-add2-4401-aceb-e7cecde3ea08"). InnerVolumeSpecName "kube-api-access-w9gbz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:26:05.079095 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.079059 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util" (OuterVolumeSpecName: "util") pod "d18608ad-add2-4401-aceb-e7cecde3ea08" (UID: "d18608ad-add2-4401-aceb-e7cecde3ea08"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 00:26:05.174609 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.174559 2575 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:05.174609 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.174579 2575 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/d18608ad-add2-4401-aceb-e7cecde3ea08-util\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:05.174609 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.174589 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9gbz\" (UniqueName: \"kubernetes.io/projected/d18608ad-add2-4401-aceb-e7cecde3ea08-kube-api-access-w9gbz\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:26:05.843767 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.843734 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" event={"ID":"d18608ad-add2-4401-aceb-e7cecde3ea08","Type":"ContainerDied","Data":"9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca"} Jul 23 00:26:05.843929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.843774 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf" Jul 23 00:26:05.843929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:05.843785 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9b2f7845ee3d940fc503ffeebeb081a67e76bb9f4b6a6cd0ad32d01dc0fcebca" Jul 23 00:26:27.653787 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.653744 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5f448b6977-jhj8l"] Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654073 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654084 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654093 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654098 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654111 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654117 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654124 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654129 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654140 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="extract" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654146 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="extract" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654154 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654159 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654168 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654174 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="pull" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654179 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654185 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="util" Jul 23 00:26:27.654185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654192 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="util" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654197 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="util" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654202 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654206 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654211 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654216 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654221 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654226 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654288 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d18608ad-add2-4401-aceb-e7cecde3ea08" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654296 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="51372e8b-ba0d-41f1-9535-706617fe2a3e" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654303 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="35c022c0-1485-4a3e-ae0b-964f4b17a244" containerName="extract" Jul 23 00:26:27.654722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.654309 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e53bc96-13d5-4716-989e-c2998b8042fd" containerName="extract" Jul 23 00:26:27.658873 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.658852 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.690372 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.690349 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f448b6977-jhj8l"] Jul 23 00:26:27.756806 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756782 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-service-ca\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.756959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756845 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pws49\" (UniqueName: \"kubernetes.io/projected/8581c2c8-5b15-41dd-a2ab-3def63863816-kube-api-access-pws49\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.756959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756871 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-trusted-ca-bundle\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.756959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756907 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-console-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.756959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756931 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-oauth-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.756959 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756952 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.757137 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.756984 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-oauth-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858304 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858273 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-console-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858426 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858312 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-oauth-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858426 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858331 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858426 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858372 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-oauth-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858426 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858410 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-service-ca\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858589 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858493 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pws49\" (UniqueName: \"kubernetes.io/projected/8581c2c8-5b15-41dd-a2ab-3def63863816-kube-api-access-pws49\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.858633 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.858615 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-trusted-ca-bundle\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.859111 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.859082 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-oauth-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.859111 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.859107 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-console-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.859475 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.859448 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-trusted-ca-bundle\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.859556 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.859481 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8581c2c8-5b15-41dd-a2ab-3def63863816-service-ca\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.860725 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.860705 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-serving-cert\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.860813 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.860796 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8581c2c8-5b15-41dd-a2ab-3def63863816-console-oauth-config\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.883659 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.883638 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pws49\" (UniqueName: \"kubernetes.io/projected/8581c2c8-5b15-41dd-a2ab-3def63863816-kube-api-access-pws49\") pod \"console-5f448b6977-jhj8l\" (UID: \"8581c2c8-5b15-41dd-a2ab-3def63863816\") " pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:27.967654 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:27.967601 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:28.098488 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:28.098461 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5f448b6977-jhj8l"] Jul 23 00:26:28.100156 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:26:28.100129 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8581c2c8_5b15_41dd_a2ab_3def63863816.slice/crio-fa0a353afb730c4c52c2a2a8e144c53c3ddfee705e0e25b4885eff929844967d WatchSource:0}: Error finding container fa0a353afb730c4c52c2a2a8e144c53c3ddfee705e0e25b4885eff929844967d: Status 404 returned error can't find the container with id fa0a353afb730c4c52c2a2a8e144c53c3ddfee705e0e25b4885eff929844967d Jul 23 00:26:28.965173 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:28.965136 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f448b6977-jhj8l" event={"ID":"8581c2c8-5b15-41dd-a2ab-3def63863816","Type":"ContainerStarted","Data":"534929baed1f36c7c27d562a0d3e487fc2f2d3e860c31885948e5b9969988386"} Jul 23 00:26:28.965589 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:28.965192 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5f448b6977-jhj8l" event={"ID":"8581c2c8-5b15-41dd-a2ab-3def63863816","Type":"ContainerStarted","Data":"fa0a353afb730c4c52c2a2a8e144c53c3ddfee705e0e25b4885eff929844967d"} Jul 23 00:26:28.985460 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:28.985414 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5f448b6977-jhj8l" podStartSLOduration=1.985398225 podStartE2EDuration="1.985398225s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:26:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:26:28.98365871 +0000 UTC m=+380.289806706" watchObservedRunningTime="2026-07-23 00:26:28.985398225 +0000 UTC m=+380.291546219" Jul 23 00:26:37.968611 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:37.968569 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:37.969005 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:37.968641 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:37.973190 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:37.973166 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:38.011670 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:38.011641 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5f448b6977-jhj8l" Jul 23 00:26:38.069164 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:26:38.069133 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:27:03.089255 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.089216 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-768cff7b7b-h677w" podUID="16896afd-31d0-48f0-959b-91b5464857dd" containerName="console" containerID="cri-o://9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b" gracePeriod=15 Jul 23 00:27:03.335670 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.335648 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:27:03.449177 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449112 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449177 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449150 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449182 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449207 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449376 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449253 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449406 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9stq\" (UniqueName: \"kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449445 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca\") pod \"16896afd-31d0-48f0-959b-91b5464857dd\" (UID: \"16896afd-31d0-48f0-959b-91b5464857dd\") " Jul 23 00:27:03.449594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449529 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:27:03.449594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449553 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config" (OuterVolumeSpecName: "console-config") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:27:03.449784 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449637 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:27:03.449930 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449907 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca" (OuterVolumeSpecName: "service-ca") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:27:03.449998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449960 2575 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-oauth-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.449998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449980 2575 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-trusted-ca-bundle\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.449998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.449994 2575 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-console-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.451421 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.451398 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:27:03.451507 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.451455 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq" (OuterVolumeSpecName: "kube-api-access-p9stq") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "kube-api-access-p9stq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:27:03.451507 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.451471 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "16896afd-31d0-48f0-959b-91b5464857dd" (UID: "16896afd-31d0-48f0-959b-91b5464857dd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:27:03.551247 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.551226 2575 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-oauth-config\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.551247 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.551247 2575 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/16896afd-31d0-48f0-959b-91b5464857dd-console-serving-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.551391 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.551256 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9stq\" (UniqueName: \"kubernetes.io/projected/16896afd-31d0-48f0-959b-91b5464857dd-kube-api-access-p9stq\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:03.551391 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:03.551265 2575 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/16896afd-31d0-48f0-959b-91b5464857dd-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:04.134461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.134423 2575 generic.go:356] "Generic (PLEG): container finished" podID="16896afd-31d0-48f0-959b-91b5464857dd" containerID="9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b" exitCode=0 Jul 23 00:27:04.134860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.134474 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-768cff7b7b-h677w" Jul 23 00:27:04.134860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.134504 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768cff7b7b-h677w" event={"ID":"16896afd-31d0-48f0-959b-91b5464857dd","Type":"ContainerDied","Data":"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b"} Jul 23 00:27:04.134860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.134559 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-768cff7b7b-h677w" event={"ID":"16896afd-31d0-48f0-959b-91b5464857dd","Type":"ContainerDied","Data":"736f1aec04ee2767e2c1c2619a08fe5a6e326c16b2cd3fed00b401ff03379980"} Jul 23 00:27:04.134860 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.134579 2575 scope.go:117] "RemoveContainer" containerID="9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b" Jul 23 00:27:04.143933 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.143917 2575 scope.go:117] "RemoveContainer" containerID="9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b" Jul 23 00:27:04.144184 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:27:04.144163 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b\": container with ID starting with 9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b not found: ID does not exist" containerID="9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b" Jul 23 00:27:04.144281 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.144189 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b"} err="failed to get container status \"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b\": rpc error: code = NotFound desc = could not find container \"9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b\": container with ID starting with 9bbb200d01cf2ecc677459ac730d9d158fc0930ae779eb4d93de88dc109aac1b not found: ID does not exist" Jul 23 00:27:04.158411 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.158387 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:27:04.162488 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:04.162469 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-768cff7b7b-h677w"] Jul 23 00:27:05.235966 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:05.235932 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16896afd-31d0-48f0-959b-91b5464857dd" path="/var/lib/kubelet/pods/16896afd-31d0-48f0-959b-91b5464857dd/volumes" Jul 23 00:27:12.460843 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.460803 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:12.461233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.461127 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16896afd-31d0-48f0-959b-91b5464857dd" containerName="console" Jul 23 00:27:12.461233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.461139 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="16896afd-31d0-48f0-959b-91b5464857dd" containerName="console" Jul 23 00:27:12.461233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.461222 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="16896afd-31d0-48f0-959b-91b5464857dd" containerName="console" Jul 23 00:27:12.465694 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.465677 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.468398 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.468369 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 23 00:27:12.468530 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.468439 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 23 00:27:12.469773 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.469750 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-6blfh\"" Jul 23 00:27:12.470262 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.469900 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 23 00:27:12.472275 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.472251 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:12.526604 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.526574 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qhgjh\" (UniqueName: \"kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.526762 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.526651 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.557842 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.557798 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:12.627737 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.627704 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.627889 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.627781 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qhgjh\" (UniqueName: \"kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.628424 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.628402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.636748 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.636726 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qhgjh\" (UniqueName: \"kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh\") pod \"limitador-limitador-64c94fd64f-glrt8\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.777543 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.777467 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:12.910309 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:12.910283 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:12.912393 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:27:12.912367 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf038d678_61d7_4901_85d8_9fe23afc7f3f.slice/crio-129516c7450cfe3f0339b83fd13a185ddc8382e9409d78366c6dce59613279df WatchSource:0}: Error finding container 129516c7450cfe3f0339b83fd13a185ddc8382e9409d78366c6dce59613279df: Status 404 returned error can't find the container with id 129516c7450cfe3f0339b83fd13a185ddc8382e9409d78366c6dce59613279df Jul 23 00:27:13.178029 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:13.177994 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" event={"ID":"f038d678-61d7-4901-85d8-9fe23afc7f3f","Type":"ContainerStarted","Data":"129516c7450cfe3f0339b83fd13a185ddc8382e9409d78366c6dce59613279df"} Jul 23 00:27:16.194899 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:16.194859 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" event={"ID":"f038d678-61d7-4901-85d8-9fe23afc7f3f","Type":"ContainerStarted","Data":"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168"} Jul 23 00:27:16.195327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:16.194944 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:16.211957 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:16.211865 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" podStartSLOduration=1.416822708 podStartE2EDuration="4.211849214s" totalImagesPullingTime="2.795026506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:27:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:27:16.211193799 +0000 UTC m=+427.517341793" watchObservedRunningTime="2026-07-23 00:27:16.211849214 +0000 UTC m=+427.517997199" Jul 23 00:27:27.200768 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:27.200736 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:29.049006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.048969 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:29.049401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.049214 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" podUID="f038d678-61d7-4901-85d8-9fe23afc7f3f" containerName="limitador" containerID="cri-o://699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168" gracePeriod=30 Jul 23 00:27:29.591155 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.591129 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:29.756472 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.756377 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file\") pod \"f038d678-61d7-4901-85d8-9fe23afc7f3f\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " Jul 23 00:27:29.756472 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.756444 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qhgjh\" (UniqueName: \"kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh\") pod \"f038d678-61d7-4901-85d8-9fe23afc7f3f\" (UID: \"f038d678-61d7-4901-85d8-9fe23afc7f3f\") " Jul 23 00:27:29.756730 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.756707 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file" (OuterVolumeSpecName: "config-file") pod "f038d678-61d7-4901-85d8-9fe23afc7f3f" (UID: "f038d678-61d7-4901-85d8-9fe23afc7f3f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:27:29.758507 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.758486 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh" (OuterVolumeSpecName: "kube-api-access-qhgjh") pod "f038d678-61d7-4901-85d8-9fe23afc7f3f" (UID: "f038d678-61d7-4901-85d8-9fe23afc7f3f"). InnerVolumeSpecName "kube-api-access-qhgjh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:27:29.857632 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.857594 2575 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/f038d678-61d7-4901-85d8-9fe23afc7f3f-config-file\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:29.857632 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:29.857627 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qhgjh\" (UniqueName: \"kubernetes.io/projected/f038d678-61d7-4901-85d8-9fe23afc7f3f-kube-api-access-qhgjh\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:27:30.262008 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.261979 2575 generic.go:356] "Generic (PLEG): container finished" podID="f038d678-61d7-4901-85d8-9fe23afc7f3f" containerID="699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168" exitCode=0 Jul 23 00:27:30.262446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.262019 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" Jul 23 00:27:30.262446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.262066 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" event={"ID":"f038d678-61d7-4901-85d8-9fe23afc7f3f","Type":"ContainerDied","Data":"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168"} Jul 23 00:27:30.262446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.262114 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-glrt8" event={"ID":"f038d678-61d7-4901-85d8-9fe23afc7f3f","Type":"ContainerDied","Data":"129516c7450cfe3f0339b83fd13a185ddc8382e9409d78366c6dce59613279df"} Jul 23 00:27:30.262446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.262140 2575 scope.go:117] "RemoveContainer" containerID="699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168" Jul 23 00:27:30.271504 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.271488 2575 scope.go:117] "RemoveContainer" containerID="699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168" Jul 23 00:27:30.271745 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:27:30.271726 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168\": container with ID starting with 699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168 not found: ID does not exist" containerID="699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168" Jul 23 00:27:30.271789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.271753 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168"} err="failed to get container status \"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168\": rpc error: code = NotFound desc = could not find container \"699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168\": container with ID starting with 699fdeaecc77316af7fa36cd46b07d212c603709bfd92c7de6e9decf76c16168 not found: ID does not exist" Jul 23 00:27:30.285195 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.285171 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:30.290878 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:30.290857 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-glrt8"] Jul 23 00:27:31.236277 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:31.236183 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f038d678-61d7-4901-85d8-9fe23afc7f3f" path="/var/lib/kubelet/pods/f038d678-61d7-4901-85d8-9fe23afc7f3f/volumes" Jul 23 00:27:34.256454 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.256417 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq"] Jul 23 00:27:34.257038 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.256745 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f038d678-61d7-4901-85d8-9fe23afc7f3f" containerName="limitador" Jul 23 00:27:34.257038 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.256756 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f038d678-61d7-4901-85d8-9fe23afc7f3f" containerName="limitador" Jul 23 00:27:34.257038 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.256811 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f038d678-61d7-4901-85d8-9fe23afc7f3f" containerName="limitador" Jul 23 00:27:34.259936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.259913 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.267675 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.267649 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-zcqq6\"" Jul 23 00:27:34.267804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.267654 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 23 00:27:34.267804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.267739 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 23 00:27:34.267986 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.267964 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq"] Jul 23 00:27:34.268626 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.268605 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 23 00:27:34.387874 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.387818 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mhpg4\" (UniqueName: \"kubernetes.io/projected/1cb17166-7167-4127-8bcd-7ff01a00a3bb-kube-api-access-mhpg4\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.388033 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.387896 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1cb17166-7167-4127-8bcd-7ff01a00a3bb-data\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.488735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.488710 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mhpg4\" (UniqueName: \"kubernetes.io/projected/1cb17166-7167-4127-8bcd-7ff01a00a3bb-kube-api-access-mhpg4\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.488911 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.488744 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1cb17166-7167-4127-8bcd-7ff01a00a3bb-data\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.489072 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.489056 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/1cb17166-7167-4127-8bcd-7ff01a00a3bb-data\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.497944 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.497915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mhpg4\" (UniqueName: \"kubernetes.io/projected/1cb17166-7167-4127-8bcd-7ff01a00a3bb-kube-api-access-mhpg4\") pod \"postgres-69fb4594cf-t2vhq\" (UID: \"1cb17166-7167-4127-8bcd-7ff01a00a3bb\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.570629 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.570608 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:34.697621 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:34.697537 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq"] Jul 23 00:27:34.700166 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:27:34.700139 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1cb17166_7167_4127_8bcd_7ff01a00a3bb.slice/crio-d345df7742e2dd3a780b73afcb9096db698d10cd3832e7cbf20b91ea3e23ace3 WatchSource:0}: Error finding container d345df7742e2dd3a780b73afcb9096db698d10cd3832e7cbf20b91ea3e23ace3: Status 404 returned error can't find the container with id d345df7742e2dd3a780b73afcb9096db698d10cd3832e7cbf20b91ea3e23ace3 Jul 23 00:27:35.285866 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:35.285811 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" event={"ID":"1cb17166-7167-4127-8bcd-7ff01a00a3bb","Type":"ContainerStarted","Data":"d345df7742e2dd3a780b73afcb9096db698d10cd3832e7cbf20b91ea3e23ace3"} Jul 23 00:27:40.314372 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:40.314340 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" event={"ID":"1cb17166-7167-4127-8bcd-7ff01a00a3bb","Type":"ContainerStarted","Data":"e7331c1d2ade45cb144af8829c6ff7b90537e9b4d64aa61b989e696120b71b3e"} Jul 23 00:27:40.314716 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:40.314487 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:40.331231 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:40.331178 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" podStartSLOduration=1.41512306 podStartE2EDuration="6.331164721s" totalImagesPullingTime="4.916041661s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:27:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:27:40.329423534 +0000 UTC m=+451.635571539" watchObservedRunningTime="2026-07-23 00:27:40.331164721 +0000 UTC m=+451.637312715" Jul 23 00:27:46.347103 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:46.347074 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-t2vhq" Jul 23 00:27:53.409732 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.409703 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:27:53.414592 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.414570 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.418498 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.418477 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 00:27:53.418498 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.418485 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 00:27:53.419348 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.419334 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h8b94\"" Jul 23 00:27:53.430575 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.430552 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:27:53.445899 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.445874 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.446006 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.445936 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5smtl\" (UniqueName: \"kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.547359 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.547332 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.547515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.547400 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5smtl\" (UniqueName: \"kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.549851 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.549808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.561063 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.561042 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5smtl\" (UniqueName: \"kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl\") pod \"maas-controller-745747874-5gt95\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.590025 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.589998 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:27:53.592336 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.592316 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.604140 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.604118 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:27:53.648400 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.648368 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.648529 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.648412 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vb6j\" (UniqueName: \"kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.725277 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.725191 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:53.749558 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.749532 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.749684 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.749582 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vb6j\" (UniqueName: \"kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.752071 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.752037 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.758766 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.758733 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vb6j\" (UniqueName: \"kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j\") pod \"maas-controller-6487687876-6vsvv\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:53.877444 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.877335 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:27:53.880221 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:27:53.880198 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf88d90c6_d121_4183_a689_e872829b0c60.slice/crio-7b2e8e8f09ef3d215ffa0657eee89ce2948f307bb44d89034b955d0e8143782c WatchSource:0}: Error finding container 7b2e8e8f09ef3d215ffa0657eee89ce2948f307bb44d89034b955d0e8143782c: Status 404 returned error can't find the container with id 7b2e8e8f09ef3d215ffa0657eee89ce2948f307bb44d89034b955d0e8143782c Jul 23 00:27:53.901730 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:53.901710 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:54.040005 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:54.039976 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:27:54.042357 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:27:54.042323 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1dc57bc5_4cb5_42d9_8ebf_d7901120156f.slice/crio-44c17d5e2b78f7c0b211b03af56ffd988eb31cdccaa57fb2dfe5f98d1ebdb2d4 WatchSource:0}: Error finding container 44c17d5e2b78f7c0b211b03af56ffd988eb31cdccaa57fb2dfe5f98d1ebdb2d4: Status 404 returned error can't find the container with id 44c17d5e2b78f7c0b211b03af56ffd988eb31cdccaa57fb2dfe5f98d1ebdb2d4 Jul 23 00:27:54.382987 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:54.382954 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-6vsvv" event={"ID":"1dc57bc5-4cb5-42d9-8ebf-d7901120156f","Type":"ContainerStarted","Data":"44c17d5e2b78f7c0b211b03af56ffd988eb31cdccaa57fb2dfe5f98d1ebdb2d4"} Jul 23 00:27:54.384087 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:54.384058 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-745747874-5gt95" event={"ID":"f88d90c6-d121-4183-a689-e872829b0c60","Type":"ContainerStarted","Data":"7b2e8e8f09ef3d215ffa0657eee89ce2948f307bb44d89034b955d0e8143782c"} Jul 23 00:27:57.402131 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.402093 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-6vsvv" event={"ID":"1dc57bc5-4cb5-42d9-8ebf-d7901120156f","Type":"ContainerStarted","Data":"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b"} Jul 23 00:27:57.402547 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.402249 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:27:57.403640 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.403616 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-745747874-5gt95" event={"ID":"f88d90c6-d121-4183-a689-e872829b0c60","Type":"ContainerStarted","Data":"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a"} Jul 23 00:27:57.403752 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.403664 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:27:57.419929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.419756 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6487687876-6vsvv" podStartSLOduration=2.012651187 podStartE2EDuration="4.419740156s" totalImagesPullingTime="2.407088969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:27:57.419159099 +0000 UTC m=+468.725307122" watchObservedRunningTime="2026-07-23 00:27:57.419740156 +0000 UTC m=+468.725888153" Jul 23 00:27:57.437534 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:27:57.437491 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-745747874-5gt95" podStartSLOduration=1.8617275 podStartE2EDuration="4.437480105s" totalImagesPullingTime="2.575752605s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:27:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:27:57.43500525 +0000 UTC m=+468.741153301" watchObservedRunningTime="2026-07-23 00:27:57.437480105 +0000 UTC m=+468.743628098" Jul 23 00:28:08.414809 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.414777 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:28:08.415293 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.414880 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:28:08.475796 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.475763 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:28:08.475973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.475954 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-745747874-5gt95" podUID="f88d90c6-d121-4183-a689-e872829b0c60" containerName="manager" containerID="cri-o://ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a" gracePeriod=30 Jul 23 00:28:08.719781 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.719756 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:28:08.768999 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.768972 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5smtl\" (UniqueName: \"kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl\") pod \"f88d90c6-d121-4183-a689-e872829b0c60\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " Jul 23 00:28:08.769158 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.769055 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert\") pod \"f88d90c6-d121-4183-a689-e872829b0c60\" (UID: \"f88d90c6-d121-4183-a689-e872829b0c60\") " Jul 23 00:28:08.771118 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.771089 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl" (OuterVolumeSpecName: "kube-api-access-5smtl") pod "f88d90c6-d121-4183-a689-e872829b0c60" (UID: "f88d90c6-d121-4183-a689-e872829b0c60"). InnerVolumeSpecName "kube-api-access-5smtl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:28:08.771229 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.771124 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f88d90c6-d121-4183-a689-e872829b0c60" (UID: "f88d90c6-d121-4183-a689-e872829b0c60"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:28:08.869662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.869634 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5smtl\" (UniqueName: \"kubernetes.io/projected/f88d90c6-d121-4183-a689-e872829b0c60-kube-api-access-5smtl\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:28:08.869662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:08.869659 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f88d90c6-d121-4183-a689-e872829b0c60-webhook-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:28:09.469243 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.469153 2575 generic.go:356] "Generic (PLEG): container finished" podID="f88d90c6-d121-4183-a689-e872829b0c60" containerID="ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a" exitCode=0 Jul 23 00:28:09.469243 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.469212 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-745747874-5gt95" Jul 23 00:28:09.469744 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.469239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-745747874-5gt95" event={"ID":"f88d90c6-d121-4183-a689-e872829b0c60","Type":"ContainerDied","Data":"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a"} Jul 23 00:28:09.469744 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.469316 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-745747874-5gt95" event={"ID":"f88d90c6-d121-4183-a689-e872829b0c60","Type":"ContainerDied","Data":"7b2e8e8f09ef3d215ffa0657eee89ce2948f307bb44d89034b955d0e8143782c"} Jul 23 00:28:09.469744 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.469337 2575 scope.go:117] "RemoveContainer" containerID="ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a" Jul 23 00:28:09.478447 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.478290 2575 scope.go:117] "RemoveContainer" containerID="ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a" Jul 23 00:28:09.478545 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:28:09.478526 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a\": container with ID starting with ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a not found: ID does not exist" containerID="ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a" Jul 23 00:28:09.478583 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.478553 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a"} err="failed to get container status \"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a\": rpc error: code = NotFound desc = could not find container \"ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a\": container with ID starting with ecab8a79030b2f31d9606880b624bcb8191d5e38ea8ddd9438de81a88f59fc3a not found: ID does not exist" Jul 23 00:28:09.486956 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.486935 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:28:09.491339 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:09.491314 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-745747874-5gt95"] Jul 23 00:28:11.237461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:11.237426 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f88d90c6-d121-4183-a689-e872829b0c60" path="/var/lib/kubelet/pods/f88d90c6-d121-4183-a689-e872829b0c60/volumes" Jul 23 00:28:29.872065 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.872027 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv"] Jul 23 00:28:29.872418 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.872353 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f88d90c6-d121-4183-a689-e872829b0c60" containerName="manager" Jul 23 00:28:29.872418 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.872364 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="f88d90c6-d121-4183-a689-e872829b0c60" containerName="manager" Jul 23 00:28:29.872488 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.872431 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="f88d90c6-d121-4183-a689-e872829b0c60" containerName="manager" Jul 23 00:28:29.900714 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.900683 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv"] Jul 23 00:28:29.900907 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.900788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:29.903235 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.903212 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 23 00:28:29.903356 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:29.903279 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-r5txj\"" Jul 23 00:28:30.037505 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.037463 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-openshift-service-ca\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.037682 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.037551 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7qrvc\" (UniqueName: \"kubernetes.io/projected/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-kube-api-access-7qrvc\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.037682 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.037582 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.138761 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.138682 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7qrvc\" (UniqueName: \"kubernetes.io/projected/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-kube-api-access-7qrvc\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.138761 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.138728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.138985 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.138778 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-openshift-service-ca\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.138985 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:28:30.138935 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 23 00:28:30.139098 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:28:30.139011 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls podName:eafbce22-d6a2-4c10-b98b-7e06c40a39ef nodeName:}" failed. No retries permitted until 2026-07-23 00:28:30.638987856 +0000 UTC m=+501.945135845 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls") pod "maas-api-8676f89cb7-bgjmv" (UID: "eafbce22-d6a2-4c10-b98b-7e06c40a39ef") : secret "maas-api-serving-cert" not found Jul 23 00:28:30.139596 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.139572 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-openshift-service-ca\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.148889 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.148865 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7qrvc\" (UniqueName: \"kubernetes.io/projected/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-kube-api-access-7qrvc\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.426141 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.426059 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wstk6"] Jul 23 00:28:30.429490 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.429469 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.430638 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.430612 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-67fgb"] Jul 23 00:28:30.432023 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.432001 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 23 00:28:30.432206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.432187 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-w2shw\"" Jul 23 00:28:30.433797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.433777 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.442814 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.442786 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wstk6"] Jul 23 00:28:30.447622 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.447598 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-67fgb"] Jul 23 00:28:30.544396 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.544368 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.544548 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.544507 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-plugins-config-volume\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.544548 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.544543 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4fll\" (UniqueName: \"kubernetes.io/projected/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-kube-api-access-t4fll\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.544663 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.544568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-69vcp\" (UniqueName: \"kubernetes.io/projected/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-kube-api-access-69vcp\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.645274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.645236 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-plugins-config-volume\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.645274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.645284 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4fll\" (UniqueName: \"kubernetes.io/projected/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-kube-api-access-t4fll\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.645504 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.645310 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-69vcp\" (UniqueName: \"kubernetes.io/projected/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-kube-api-access-69vcp\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.645504 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.645348 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.645504 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.645422 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.646238 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.646206 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-plugins-config-volume\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.646349 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.646214 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.648553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.648532 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/eafbce22-d6a2-4c10-b98b-7e06c40a39ef-maas-api-tls\") pod \"maas-api-8676f89cb7-bgjmv\" (UID: \"eafbce22-d6a2-4c10-b98b-7e06c40a39ef\") " pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.654362 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.654338 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-69vcp\" (UniqueName: \"kubernetes.io/projected/80798469-9bb3-409e-8b9e-9d4fe1b2c1e3-kube-api-access-69vcp\") pod \"payload-pre-processing-7677b774bb-wstk6\" (UID: \"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.654597 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.654508 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4fll\" (UniqueName: \"kubernetes.io/projected/f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89-kube-api-access-t4fll\") pod \"payload-processing-5bf47cd988-67fgb\" (UID: \"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89\") " pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.743332 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.743283 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:30.752538 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.752494 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:30.812942 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.812905 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:30.891394 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.890095 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-wstk6"] Jul 23 00:28:30.896135 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:28:30.896062 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80798469_9bb3_409e_8b9e_9d4fe1b2c1e3.slice/crio-85d04b6ef1f2eff24b80905e5cf6de126dad2137944774468e05c2c106cd2c25 WatchSource:0}: Error finding container 85d04b6ef1f2eff24b80905e5cf6de126dad2137944774468e05c2c106cd2c25: Status 404 returned error can't find the container with id 85d04b6ef1f2eff24b80905e5cf6de126dad2137944774468e05c2c106cd2c25 Jul 23 00:28:30.917253 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.917214 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5bf47cd988-67fgb"] Jul 23 00:28:30.918997 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:28:30.918967 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3633e4b_62b9_4b7f_9cb1_3da6b2a74c89.slice/crio-087a05e1bb0fff6568ea9811d34b05c0da5fd609eeeec5c41bf075d41160f948 WatchSource:0}: Error finding container 087a05e1bb0fff6568ea9811d34b05c0da5fd609eeeec5c41bf075d41160f948: Status 404 returned error can't find the container with id 087a05e1bb0fff6568ea9811d34b05c0da5fd609eeeec5c41bf075d41160f948 Jul 23 00:28:30.970914 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:30.970888 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv"] Jul 23 00:28:30.972414 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:28:30.972389 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeafbce22_d6a2_4c10_b98b_7e06c40a39ef.slice/crio-b808b0a93bc6842394b3c5adf17626b31e555d344266b195c2a36ea8734a00eb WatchSource:0}: Error finding container b808b0a93bc6842394b3c5adf17626b31e555d344266b195c2a36ea8734a00eb: Status 404 returned error can't find the container with id b808b0a93bc6842394b3c5adf17626b31e555d344266b195c2a36ea8734a00eb Jul 23 00:28:31.600629 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:31.600541 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" event={"ID":"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89","Type":"ContainerStarted","Data":"087a05e1bb0fff6568ea9811d34b05c0da5fd609eeeec5c41bf075d41160f948"} Jul 23 00:28:31.605240 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:31.605119 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" event={"ID":"eafbce22-d6a2-4c10-b98b-7e06c40a39ef","Type":"ContainerStarted","Data":"b808b0a93bc6842394b3c5adf17626b31e555d344266b195c2a36ea8734a00eb"} Jul 23 00:28:31.607856 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:31.607805 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" event={"ID":"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3","Type":"ContainerStarted","Data":"85d04b6ef1f2eff24b80905e5cf6de126dad2137944774468e05c2c106cd2c25"} Jul 23 00:28:33.619350 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.619313 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" event={"ID":"f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89","Type":"ContainerStarted","Data":"1f22222840f49fe7f88d81671ddee7846735148f48a63f47d9698a0b18526dd0"} Jul 23 00:28:33.619772 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.619385 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:33.620726 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.620703 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" event={"ID":"eafbce22-d6a2-4c10-b98b-7e06c40a39ef","Type":"ContainerStarted","Data":"10c91ff957bbaf41fae419b59d177e1720e1809c7c14ab1ee91873e5578d6228"} Jul 23 00:28:33.620875 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.620796 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:33.622125 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.622099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" event={"ID":"80798469-9bb3-409e-8b9e-9d4fe1b2c1e3","Type":"ContainerStarted","Data":"3f1e205ae303e9eaebac7d7b53440fa55c71ff3e356dd2ec99f4dd2298fec78b"} Jul 23 00:28:33.622236 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.622218 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:33.635743 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.635700 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" podStartSLOduration=1.444038242 podStartE2EDuration="3.635688194s" totalImagesPullingTime="2.191649952s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:33.634738944 +0000 UTC m=+504.940886949" watchObservedRunningTime="2026-07-23 00:28:33.635688194 +0000 UTC m=+504.941836187" Jul 23 00:28:33.650690 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.650646 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" podStartSLOduration=2.51727686 podStartE2EDuration="4.650632876s" totalImagesPullingTime="2.133356016s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:28:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:33.649489064 +0000 UTC m=+504.955637061" watchObservedRunningTime="2026-07-23 00:28:33.650632876 +0000 UTC m=+504.956780869" Jul 23 00:28:33.665044 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:33.664995 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" podStartSLOduration=1.4561913149999999 podStartE2EDuration="3.664980382s" totalImagesPullingTime="2.208789067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:28:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:33.664536891 +0000 UTC m=+504.970684903" watchObservedRunningTime="2026-07-23 00:28:33.664980382 +0000 UTC m=+504.971128376" Jul 23 00:28:39.633679 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:39.633649 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-8676f89cb7-bgjmv" Jul 23 00:28:44.630407 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:44.630376 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5bf47cd988-67fgb" Jul 23 00:28:44.630885 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:44.630442 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-wstk6" Jul 23 00:28:49.161436 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.161398 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp"] Jul 23 00:28:49.165161 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.165141 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.168874 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.168851 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 23 00:28:49.169003 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.168947 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 23 00:28:49.169003 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.168971 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4vnxc\"" Jul 23 00:28:49.169479 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.169462 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 23 00:28:49.174395 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.174374 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp"] Jul 23 00:28:49.300325 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300300 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.300484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300331 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb645107-c098-444c-9c68-6b796dc46f1d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.300484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300361 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.300484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300422 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.300484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300439 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.300484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.300465 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kxn79\" (UniqueName: \"kubernetes.io/projected/eb645107-c098-444c-9c68-6b796dc46f1d-kube-api-access-kxn79\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.401875 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.401824 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kxn79\" (UniqueName: \"kubernetes.io/projected/eb645107-c098-444c-9c68-6b796dc46f1d-kube-api-access-kxn79\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402004 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.401963 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402004 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.401985 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb645107-c098-444c-9c68-6b796dc46f1d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402075 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402014 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402075 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402057 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402150 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402076 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402416 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402393 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402478 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402402 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.402478 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.402461 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.404319 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.404300 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb645107-c098-444c-9c68-6b796dc46f1d-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.404435 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.404419 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb645107-c098-444c-9c68-6b796dc46f1d-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.410937 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.410915 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kxn79\" (UniqueName: \"kubernetes.io/projected/eb645107-c098-444c-9c68-6b796dc46f1d-kube-api-access-kxn79\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp\" (UID: \"eb645107-c098-444c-9c68-6b796dc46f1d\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.477031 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.476979 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:49.606408 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.606381 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp"] Jul 23 00:28:49.607587 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:28:49.607561 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb645107_c098_444c_9c68_6b796dc46f1d.slice/crio-ba4e5aa6eb6d756557d3beb5f2fe8e41c7e122996623779a8933bacbb7c35253 WatchSource:0}: Error finding container ba4e5aa6eb6d756557d3beb5f2fe8e41c7e122996623779a8933bacbb7c35253: Status 404 returned error can't find the container with id ba4e5aa6eb6d756557d3beb5f2fe8e41c7e122996623779a8933bacbb7c35253 Jul 23 00:28:49.707416 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:49.707390 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" event={"ID":"eb645107-c098-444c-9c68-6b796dc46f1d","Type":"ContainerStarted","Data":"ba4e5aa6eb6d756557d3beb5f2fe8e41c7e122996623779a8933bacbb7c35253"} Jul 23 00:28:50.714719 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:50.714687 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" event={"ID":"eb645107-c098-444c-9c68-6b796dc46f1d","Type":"ContainerStarted","Data":"61b0d9587a2d5ab3b05185d7e2fa2d12f03e3014e55e2fae8b15c94a3b8a34e4"} Jul 23 00:28:50.732885 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:50.732820 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" podStartSLOduration=0.716276338 podStartE2EDuration="1.732805638s" totalImagesPullingTime="1.0165293s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:28:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:28:50.730600209 +0000 UTC m=+522.036748229" watchObservedRunningTime="2026-07-23 00:28:50.732805638 +0000 UTC m=+522.038953631" Jul 23 00:28:59.477673 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:59.477635 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:59.478091 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:59.477703 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:59.490167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:59.490143 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:28:59.773768 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:28:59.773697 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp" Jul 23 00:29:08.853929 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.853856 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4"] Jul 23 00:29:08.856958 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.856936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.859794 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.859774 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 00:29:08.868440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.868420 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4"] Jul 23 00:29:08.954892 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.954869 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.955007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.954897 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.955007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.954914 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ac46ac-6046-46dc-9670-dc7b1c534924-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.955007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.954937 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.955007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.954987 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:08.955143 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:08.955066 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5rkf\" (UniqueName: \"kubernetes.io/projected/70ac46ac-6046-46dc-9670-dc7b1c534924-kube-api-access-r5rkf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055384 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055356 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055384 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055388 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055596 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055405 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ac46ac-6046-46dc-9670-dc7b1c534924-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055596 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055425 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055596 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055546 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055750 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055622 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5rkf\" (UniqueName: \"kubernetes.io/projected/70ac46ac-6046-46dc-9670-dc7b1c534924-kube-api-access-r5rkf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055883 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055862 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055890 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.055947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.055892 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.057493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.057476 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/70ac46ac-6046-46dc-9670-dc7b1c534924-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.058176 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.058161 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 23 00:29:09.063869 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.063823 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5rkf\" (UniqueName: \"kubernetes.io/projected/70ac46ac-6046-46dc-9670-dc7b1c534924-kube-api-access-r5rkf\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.068563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.068534 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/70ac46ac-6046-46dc-9670-dc7b1c534924-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-hwkq4\" (UID: \"70ac46ac-6046-46dc-9670-dc7b1c534924\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.168118 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.168094 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:09.307152 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.307118 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4"] Jul 23 00:29:09.308658 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:29:09.308635 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70ac46ac_6046_46dc_9670_dc7b1c534924.slice/crio-07c47bc05ade94286e7c86ef4a0839fef1e8e8b34af77fcf2156be50420a7466 WatchSource:0}: Error finding container 07c47bc05ade94286e7c86ef4a0839fef1e8e8b34af77fcf2156be50420a7466: Status 404 returned error can't find the container with id 07c47bc05ade94286e7c86ef4a0839fef1e8e8b34af77fcf2156be50420a7466 Jul 23 00:29:09.823589 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.823547 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" event={"ID":"70ac46ac-6046-46dc-9670-dc7b1c534924","Type":"ContainerStarted","Data":"23ff86486092e06005a40557476bdee296a62e18bfca01d8929156275f08789d"} Jul 23 00:29:09.823863 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.823637 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" event={"ID":"70ac46ac-6046-46dc-9670-dc7b1c534924","Type":"ContainerStarted","Data":"07c47bc05ade94286e7c86ef4a0839fef1e8e8b34af77fcf2156be50420a7466"} Jul 23 00:29:09.842473 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:09.842420 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" podStartSLOduration=1.652503525 podStartE2EDuration="1.842400993s" totalImagesPullingTime="189.897468ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:29:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:29:09.839570623 +0000 UTC m=+541.145718630" watchObservedRunningTime="2026-07-23 00:29:09.842400993 +0000 UTC m=+541.148548989" Jul 23 00:29:19.168663 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:19.168627 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:19.169226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:19.168711 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:19.181402 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:19.181376 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:19.884843 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:19.884798 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-hwkq4" Jul 23 00:29:35.557555 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.557520 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c"] Jul 23 00:29:35.561596 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.561575 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.564065 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.564044 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 23 00:29:35.568618 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.568594 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c"] Jul 23 00:29:35.686189 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686156 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.686189 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686191 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab34665-c096-4644-86de-7183825ce4fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.686373 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686211 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9cxpx\" (UniqueName: \"kubernetes.io/projected/dab34665-c096-4644-86de-7183825ce4fb-kube-api-access-9cxpx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.686373 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686302 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.686373 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686336 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.686477 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.686411 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787010 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.786979 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787010 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787013 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787230 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787047 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787230 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787100 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787230 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787117 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab34665-c096-4644-86de-7183825ce4fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787230 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787134 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9cxpx\" (UniqueName: \"kubernetes.io/projected/dab34665-c096-4644-86de-7183825ce4fb-kube-api-access-9cxpx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787439 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787401 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787495 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.787603 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.787585 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.789393 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.789373 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dab34665-c096-4644-86de-7183825ce4fb-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.789553 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.789535 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dab34665-c096-4644-86de-7183825ce4fb-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.794765 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.794741 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9cxpx\" (UniqueName: \"kubernetes.io/projected/dab34665-c096-4644-86de-7183825ce4fb-kube-api-access-9cxpx\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5qg4c\" (UID: \"dab34665-c096-4644-86de-7183825ce4fb\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:35.873870 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:35.873788 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:36.001025 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:36.000999 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c"] Jul 23 00:29:36.002609 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:29:36.002585 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddab34665_c096_4644_86de_7183825ce4fb.slice/crio-e4f76374d267d6df0efca72a6e9bd56e35accc9abac47f9707b1ac26220d2ae5 WatchSource:0}: Error finding container e4f76374d267d6df0efca72a6e9bd56e35accc9abac47f9707b1ac26220d2ae5: Status 404 returned error can't find the container with id e4f76374d267d6df0efca72a6e9bd56e35accc9abac47f9707b1ac26220d2ae5 Jul 23 00:29:36.969088 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:36.969049 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" event={"ID":"dab34665-c096-4644-86de-7183825ce4fb","Type":"ContainerStarted","Data":"80f4c810c4040ee5514bbe3cebef8576cce08ff30ca1ec539636b6c1be1b475c"} Jul 23 00:29:36.969453 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:36.969113 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" event={"ID":"dab34665-c096-4644-86de-7183825ce4fb","Type":"ContainerStarted","Data":"e4f76374d267d6df0efca72a6e9bd56e35accc9abac47f9707b1ac26220d2ae5"} Jul 23 00:29:36.990592 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:36.990535 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" podStartSLOduration=1.8225581640000001 podStartE2EDuration="1.990519115s" totalImagesPullingTime="167.960951ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:29:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:29:36.987447041 +0000 UTC m=+568.293595052" watchObservedRunningTime="2026-07-23 00:29:36.990519115 +0000 UTC m=+568.296667108" Jul 23 00:29:45.875016 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:45.874973 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:45.875449 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:45.875036 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:45.887326 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:45.887300 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:29:46.026575 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:29:46.026550 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5qg4c" Jul 23 00:30:09.107737 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:30:09.107708 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:30:09.109443 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:30:09.109420 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:31:59.706107 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.706071 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:31:59.706595 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.706267 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6487687876-6vsvv" podUID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" containerName="manager" containerID="cri-o://e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b" gracePeriod=30 Jul 23 00:31:59.948311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.948285 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:31:59.994468 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.994405 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vb6j\" (UniqueName: \"kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j\") pod \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " Jul 23 00:31:59.994468 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.994454 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert\") pod \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\" (UID: \"1dc57bc5-4cb5-42d9-8ebf-d7901120156f\") " Jul 23 00:31:59.996377 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.996350 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j" (OuterVolumeSpecName: "kube-api-access-5vb6j") pod "1dc57bc5-4cb5-42d9-8ebf-d7901120156f" (UID: "1dc57bc5-4cb5-42d9-8ebf-d7901120156f"). InnerVolumeSpecName "kube-api-access-5vb6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:31:59.996468 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:31:59.996400 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "1dc57bc5-4cb5-42d9-8ebf-d7901120156f" (UID: "1dc57bc5-4cb5-42d9-8ebf-d7901120156f"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:32:00.095228 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.095198 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vb6j\" (UniqueName: \"kubernetes.io/projected/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-kube-api-access-5vb6j\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:32:00.095228 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.095226 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/1dc57bc5-4cb5-42d9-8ebf-d7901120156f-webhook-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:32:00.713847 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.713745 2575 generic.go:356] "Generic (PLEG): container finished" podID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" containerID="e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b" exitCode=0 Jul 23 00:32:00.713847 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.713783 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-6vsvv" Jul 23 00:32:00.713847 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.713804 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-6vsvv" event={"ID":"1dc57bc5-4cb5-42d9-8ebf-d7901120156f","Type":"ContainerDied","Data":"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b"} Jul 23 00:32:00.714380 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.713885 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-6vsvv" event={"ID":"1dc57bc5-4cb5-42d9-8ebf-d7901120156f","Type":"ContainerDied","Data":"44c17d5e2b78f7c0b211b03af56ffd988eb31cdccaa57fb2dfe5f98d1ebdb2d4"} Jul 23 00:32:00.714380 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.713917 2575 scope.go:117] "RemoveContainer" containerID="e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b" Jul 23 00:32:00.723050 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.723034 2575 scope.go:117] "RemoveContainer" containerID="e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b" Jul 23 00:32:00.723307 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:32:00.723281 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b\": container with ID starting with e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b not found: ID does not exist" containerID="e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b" Jul 23 00:32:00.723375 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.723310 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b"} err="failed to get container status \"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b\": rpc error: code = NotFound desc = could not find container \"e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b\": container with ID starting with e626140bc0f64634e39cf895c651638ea516b3f3d72b5c783aaf44dbfdced25b not found: ID does not exist" Jul 23 00:32:00.735516 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.735490 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:32:00.738652 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:00.738631 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6487687876-6vsvv"] Jul 23 00:32:01.236300 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:32:01.236258 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" path="/var/lib/kubelet/pods/1dc57bc5-4cb5-42d9-8ebf-d7901120156f/volumes" Jul 23 00:33:01.609123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.609090 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:01.609662 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.609641 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" containerName="manager" Jul 23 00:33:01.609743 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.609665 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" containerName="manager" Jul 23 00:33:01.609795 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.609762 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1dc57bc5-4cb5-42d9-8ebf-d7901120156f" containerName="manager" Jul 23 00:33:01.612787 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.612768 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.616538 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.616517 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h8b94\"" Jul 23 00:33:01.617572 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.617556 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 00:33:01.625797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.625778 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 00:33:01.665167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.665137 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:01.689552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.689521 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tjk5f\" (UniqueName: \"kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.689756 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.689640 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.791276 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.791243 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.791446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.791351 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tjk5f\" (UniqueName: \"kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.793722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.793703 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.806540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.806518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tjk5f\" (UniqueName: \"kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f\") pod \"maas-controller-6487687876-vjlln\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:01.924378 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:01.924286 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:02.055513 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:02.055487 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:02.056435 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:33:02.056404 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3ebee32_6b15_4008_9977_d7d4162eaa08.slice/crio-3a450ee97236641e210322e3146153d176d417c3807bf5c079ce3a08caacdd8f WatchSource:0}: Error finding container 3a450ee97236641e210322e3146153d176d417c3807bf5c079ce3a08caacdd8f: Status 404 returned error can't find the container with id 3a450ee97236641e210322e3146153d176d417c3807bf5c079ce3a08caacdd8f Jul 23 00:33:02.057695 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:02.057676 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:33:03.038944 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:03.038906 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-vjlln" event={"ID":"d3ebee32-6b15-4008-9977-d7d4162eaa08","Type":"ContainerStarted","Data":"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a"} Jul 23 00:33:03.039355 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:03.038967 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-vjlln" event={"ID":"d3ebee32-6b15-4008-9977-d7d4162eaa08","Type":"ContainerStarted","Data":"3a450ee97236641e210322e3146153d176d417c3807bf5c079ce3a08caacdd8f"} Jul 23 00:33:03.039355 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:03.039011 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:03.059238 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:03.059184 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6487687876-vjlln" podStartSLOduration=1.608941025 podStartE2EDuration="2.059169823s" totalImagesPullingTime="450.228798ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:03.056538377 +0000 UTC m=+774.362686389" watchObservedRunningTime="2026-07-23 00:33:03.059169823 +0000 UTC m=+774.365317817" Jul 23 00:33:14.046801 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:14.046769 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:18.708107 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:18.708071 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:18.708594 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:18.708291 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6487687876-vjlln" podUID="d3ebee32-6b15-4008-9977-d7d4162eaa08" containerName="manager" containerID="cri-o://61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a" gracePeriod=30 Jul 23 00:33:18.951904 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:18.951882 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:19.032746 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.032683 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tjk5f\" (UniqueName: \"kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f\") pod \"d3ebee32-6b15-4008-9977-d7d4162eaa08\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " Jul 23 00:33:19.032746 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.032721 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert\") pod \"d3ebee32-6b15-4008-9977-d7d4162eaa08\" (UID: \"d3ebee32-6b15-4008-9977-d7d4162eaa08\") " Jul 23 00:33:19.034639 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.034616 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "d3ebee32-6b15-4008-9977-d7d4162eaa08" (UID: "d3ebee32-6b15-4008-9977-d7d4162eaa08"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:33:19.034722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.034707 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f" (OuterVolumeSpecName: "kube-api-access-tjk5f") pod "d3ebee32-6b15-4008-9977-d7d4162eaa08" (UID: "d3ebee32-6b15-4008-9977-d7d4162eaa08"). InnerVolumeSpecName "kube-api-access-tjk5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:33:19.125228 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.125194 2575 generic.go:356] "Generic (PLEG): container finished" podID="d3ebee32-6b15-4008-9977-d7d4162eaa08" containerID="61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a" exitCode=0 Jul 23 00:33:19.125360 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.125243 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-vjlln" Jul 23 00:33:19.125360 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.125282 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-vjlln" event={"ID":"d3ebee32-6b15-4008-9977-d7d4162eaa08","Type":"ContainerDied","Data":"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a"} Jul 23 00:33:19.125360 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.125336 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-vjlln" event={"ID":"d3ebee32-6b15-4008-9977-d7d4162eaa08","Type":"ContainerDied","Data":"3a450ee97236641e210322e3146153d176d417c3807bf5c079ce3a08caacdd8f"} Jul 23 00:33:19.125511 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.125364 2575 scope.go:117] "RemoveContainer" containerID="61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a" Jul 23 00:33:19.133456 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.133436 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tjk5f\" (UniqueName: \"kubernetes.io/projected/d3ebee32-6b15-4008-9977-d7d4162eaa08-kube-api-access-tjk5f\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:33:19.133543 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.133459 2575 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/d3ebee32-6b15-4008-9977-d7d4162eaa08-webhook-cert\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:33:19.134292 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.134278 2575 scope.go:117] "RemoveContainer" containerID="61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a" Jul 23 00:33:19.134523 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:33:19.134505 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a\": container with ID starting with 61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a not found: ID does not exist" containerID="61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a" Jul 23 00:33:19.134563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.134531 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a"} err="failed to get container status \"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a\": rpc error: code = NotFound desc = could not find container \"61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a\": container with ID starting with 61f87b3ac0d6813c473f2af25c20e7bf377aae27f8cb8af58a384d3bf92e403a not found: ID does not exist" Jul 23 00:33:19.146688 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.146664 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:19.150573 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.150551 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6487687876-vjlln"] Jul 23 00:33:19.236791 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.236769 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3ebee32-6b15-4008-9977-d7d4162eaa08" path="/var/lib/kubelet/pods/d3ebee32-6b15-4008-9977-d7d4162eaa08/volumes" Jul 23 00:33:19.976739 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.976699 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6487687876-pk67l"] Jul 23 00:33:19.977247 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.977228 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3ebee32-6b15-4008-9977-d7d4162eaa08" containerName="manager" Jul 23 00:33:19.977327 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.977251 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3ebee32-6b15-4008-9977-d7d4162eaa08" containerName="manager" Jul 23 00:33:19.977392 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.977378 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3ebee32-6b15-4008-9977-d7d4162eaa08" containerName="manager" Jul 23 00:33:19.981549 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.981529 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:19.985104 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.985084 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-h8b94\"" Jul 23 00:33:19.985223 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.985086 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 23 00:33:19.985223 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.985123 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 23 00:33:19.991026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:19.991002 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-pk67l"] Jul 23 00:33:20.140558 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.140516 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l6nkd\" (UniqueName: \"kubernetes.io/projected/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-kube-api-access-l6nkd\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.140714 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.140579 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-webhook-cert\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.241349 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.241292 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l6nkd\" (UniqueName: \"kubernetes.io/projected/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-kube-api-access-l6nkd\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.241349 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.241329 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-webhook-cert\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.243536 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.243518 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-webhook-cert\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.251677 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.251655 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l6nkd\" (UniqueName: \"kubernetes.io/projected/38933b96-ddd8-4778-ab7d-a8e4486ffcb4-kube-api-access-l6nkd\") pod \"maas-controller-6487687876-pk67l\" (UID: \"38933b96-ddd8-4778-ab7d-a8e4486ffcb4\") " pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.292623 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.292596 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:20.419345 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:20.419322 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6487687876-pk67l"] Jul 23 00:33:20.421124 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:33:20.421096 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38933b96_ddd8_4778_ab7d_a8e4486ffcb4.slice/crio-6c1c06b2294e063d552193212d6f73b3637438f6d79bf382f59b077b6050057a WatchSource:0}: Error finding container 6c1c06b2294e063d552193212d6f73b3637438f6d79bf382f59b077b6050057a: Status 404 returned error can't find the container with id 6c1c06b2294e063d552193212d6f73b3637438f6d79bf382f59b077b6050057a Jul 23 00:33:21.137515 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:21.137486 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-pk67l" event={"ID":"38933b96-ddd8-4778-ab7d-a8e4486ffcb4","Type":"ContainerStarted","Data":"092cc5817898ffd79bab5badb514f50fc2e12c6b7639eb3e901a96f019678fde"} Jul 23 00:33:21.137811 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:21.137576 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6487687876-pk67l" event={"ID":"38933b96-ddd8-4778-ab7d-a8e4486ffcb4","Type":"ContainerStarted","Data":"6c1c06b2294e063d552193212d6f73b3637438f6d79bf382f59b077b6050057a"} Jul 23 00:33:21.137811 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:21.137601 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:33:21.156484 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:21.156437 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6487687876-pk67l" podStartSLOduration=1.52201987 podStartE2EDuration="2.156423238s" totalImagesPullingTime="634.403368ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:33:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:33:21.154068701 +0000 UTC m=+792.460216694" watchObservedRunningTime="2026-07-23 00:33:21.156423238 +0000 UTC m=+792.462571257" Jul 23 00:33:32.145924 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:33:32.145847 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6487687876-pk67l" Jul 23 00:35:09.143942 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:35:09.143762 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:35:09.146122 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:35:09.146096 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:40:09.179773 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:40:09.179742 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:40:09.184289 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:40:09.184266 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:45:09.216530 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:45:09.216500 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:45:09.221021 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:45:09.220999 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:46:17.609117 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.609081 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:17.612683 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.612664 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.615183 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.615161 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-0b6bf1ce-serving-cert\"" Jul 23 00:46:17.621633 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.621614 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:17.790739 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.790702 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.790739 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.790742 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfdpj\" (UniqueName: \"kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.790980 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.790851 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.892037 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.891958 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.892180 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.892039 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.892180 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.892075 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfdpj\" (UniqueName: \"kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.892180 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:46:17.892114 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-serving-cert: secret "maas-api-e2e-ait-0b6bf1ce-serving-cert" not found Jul 23 00:46:17.892180 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:46:17.892175 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls podName:6cd802ad-4702-4ee2-9061-4c7614a928a7 nodeName:}" failed. No retries permitted until 2026-07-23 00:46:18.392159228 +0000 UTC m=+1569.698307200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls") pod "maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" (UID: "6cd802ad-4702-4ee2-9061-4c7614a928a7") : secret "maas-api-e2e-ait-0b6bf1ce-serving-cert" not found Jul 23 00:46:17.892634 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.892614 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:17.900900 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:17.900877 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfdpj\" (UniqueName: \"kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:18.396923 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:18.396890 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:18.397104 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:46:18.397068 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-serving-cert: secret "maas-api-e2e-ait-0b6bf1ce-serving-cert" not found Jul 23 00:46:18.397173 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:46:18.397152 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls podName:6cd802ad-4702-4ee2-9061-4c7614a928a7 nodeName:}" failed. No retries permitted until 2026-07-23 00:46:19.397131363 +0000 UTC m=+1570.703279334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls") pod "maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" (UID: "6cd802ad-4702-4ee2-9061-4c7614a928a7") : secret "maas-api-e2e-ait-0b6bf1ce-serving-cert" not found Jul 23 00:46:19.406749 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:19.406712 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:19.409131 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:19.409107 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:19.424078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:19.424053 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:19.562290 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:19.562260 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:19.564365 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:46:19.564336 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6cd802ad_4702_4ee2_9061_4c7614a928a7.slice/crio-26b139abbb3a14858fb77555fe5aa3951e8769107d0516c475cf6aff8fa4f4e7 WatchSource:0}: Error finding container 26b139abbb3a14858fb77555fe5aa3951e8769107d0516c475cf6aff8fa4f4e7: Status 404 returned error can't find the container with id 26b139abbb3a14858fb77555fe5aa3951e8769107d0516c475cf6aff8fa4f4e7 Jul 23 00:46:19.565804 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:19.565783 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:46:20.142730 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:20.142698 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" event={"ID":"6cd802ad-4702-4ee2-9061-4c7614a928a7","Type":"ContainerStarted","Data":"26b139abbb3a14858fb77555fe5aa3951e8769107d0516c475cf6aff8fa4f4e7"} Jul 23 00:46:21.149868 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:21.149807 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" event={"ID":"6cd802ad-4702-4ee2-9061-4c7614a928a7","Type":"ContainerStarted","Data":"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0"} Jul 23 00:46:21.150232 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:21.149895 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:21.167541 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:21.167492 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" podStartSLOduration=3.648591761 podStartE2EDuration="4.167476246s" totalImagesPullingTime="518.884485ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:46:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:46:21.164673497 +0000 UTC m=+1572.470821491" watchObservedRunningTime="2026-07-23 00:46:21.167476246 +0000 UTC m=+1572.473624260" Jul 23 00:46:27.159981 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:27.159953 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:53.661781 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.661744 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:53.662382 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.662023 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" podUID="6cd802ad-4702-4ee2-9061-4c7614a928a7" containerName="maas-api" containerID="cri-o://3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0" gracePeriod=30 Jul 23 00:46:53.912440 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.912387 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:53.990836 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.990806 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bfdpj\" (UniqueName: \"kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj\") pod \"6cd802ad-4702-4ee2-9061-4c7614a928a7\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " Jul 23 00:46:53.990950 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.990866 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") pod \"6cd802ad-4702-4ee2-9061-4c7614a928a7\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " Jul 23 00:46:53.990950 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.990889 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca\") pod \"6cd802ad-4702-4ee2-9061-4c7614a928a7\" (UID: \"6cd802ad-4702-4ee2-9061-4c7614a928a7\") " Jul 23 00:46:53.991323 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.991301 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6cd802ad-4702-4ee2-9061-4c7614a928a7" (UID: "6cd802ad-4702-4ee2-9061-4c7614a928a7"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:46:53.993044 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.993020 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6cd802ad-4702-4ee2-9061-4c7614a928a7" (UID: "6cd802ad-4702-4ee2-9061-4c7614a928a7"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:46:53.993106 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:53.993052 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj" (OuterVolumeSpecName: "kube-api-access-bfdpj") pod "6cd802ad-4702-4ee2-9061-4c7614a928a7" (UID: "6cd802ad-4702-4ee2-9061-4c7614a928a7"). InnerVolumeSpecName "kube-api-access-bfdpj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:46:54.091784 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.091757 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bfdpj\" (UniqueName: \"kubernetes.io/projected/6cd802ad-4702-4ee2-9061-4c7614a928a7-kube-api-access-bfdpj\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:46:54.091784 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.091781 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6cd802ad-4702-4ee2-9061-4c7614a928a7-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:46:54.091939 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.091792 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6cd802ad-4702-4ee2-9061-4c7614a928a7-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:46:54.321948 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.321913 2575 generic.go:356] "Generic (PLEG): container finished" podID="6cd802ad-4702-4ee2-9061-4c7614a928a7" containerID="3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0" exitCode=0 Jul 23 00:46:54.322123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.321967 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" Jul 23 00:46:54.322123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.322004 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" event={"ID":"6cd802ad-4702-4ee2-9061-4c7614a928a7","Type":"ContainerDied","Data":"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0"} Jul 23 00:46:54.322123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.322077 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq" event={"ID":"6cd802ad-4702-4ee2-9061-4c7614a928a7","Type":"ContainerDied","Data":"26b139abbb3a14858fb77555fe5aa3951e8769107d0516c475cf6aff8fa4f4e7"} Jul 23 00:46:54.322123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.322107 2575 scope.go:117] "RemoveContainer" containerID="3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0" Jul 23 00:46:54.338353 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.338325 2575 scope.go:117] "RemoveContainer" containerID="3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0" Jul 23 00:46:54.338641 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:46:54.338614 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0\": container with ID starting with 3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0 not found: ID does not exist" containerID="3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0" Jul 23 00:46:54.338713 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.338654 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0"} err="failed to get container status \"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0\": rpc error: code = NotFound desc = could not find container \"3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0\": container with ID starting with 3961dd38f63dcc764c4991df00afbed184c8c83923e87611f8247cbc79621aa0 not found: ID does not exist" Jul 23 00:46:54.349162 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.349140 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:54.353499 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:54.353478 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-0b6bf1ce-959b8765f-5c5cq"] Jul 23 00:46:55.237181 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:46:55.237145 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6cd802ad-4702-4ee2-9061-4c7614a928a7" path="/var/lib/kubelet/pods/6cd802ad-4702-4ee2-9061-4c7614a928a7/volumes" Jul 23 00:47:02.006819 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.006744 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:02.007167 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.007154 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6cd802ad-4702-4ee2-9061-4c7614a928a7" containerName="maas-api" Jul 23 00:47:02.007218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.007169 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6cd802ad-4702-4ee2-9061-4c7614a928a7" containerName="maas-api" Jul 23 00:47:02.007257 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.007242 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6cd802ad-4702-4ee2-9061-4c7614a928a7" containerName="maas-api" Jul 23 00:47:02.011601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.011585 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.015301 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.015274 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-migrate-17c44e03-serving-cert\"" Jul 23 00:47:02.027282 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.027260 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:02.159070 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.158928 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.159233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.159178 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.159233 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.159215 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmvp9\" (UniqueName: \"kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.260156 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.260084 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.260156 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.260141 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmvp9\" (UniqueName: \"kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.260320 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:47:02.260211 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-serving-cert: secret "maas-api-e2e-migrate-17c44e03-serving-cert" not found Jul 23 00:47:02.260320 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.260246 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.260320 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:47:02.260286 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls podName:ab16a327-daf8-4f11-ad99-a97ca6d3380b nodeName:}" failed. No retries permitted until 2026-07-23 00:47:02.760270175 +0000 UTC m=+1614.066418151 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls") pod "maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" (UID: "ab16a327-daf8-4f11-ad99-a97ca6d3380b") : secret "maas-api-e2e-migrate-17c44e03-serving-cert" not found Jul 23 00:47:02.260846 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.260808 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.268363 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.268341 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmvp9\" (UniqueName: \"kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.763646 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.763609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.766110 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.766085 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") pod \"maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:02.921900 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:02.921866 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:03.058205 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:03.058179 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:03.060287 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:47:03.060253 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab16a327_daf8_4f11_ad99_a97ca6d3380b.slice/crio-bdf5f33c3b582f709363748d41e16c9157c104d074cc160a9923aa5fc962c220 WatchSource:0}: Error finding container bdf5f33c3b582f709363748d41e16c9157c104d074cc160a9923aa5fc962c220: Status 404 returned error can't find the container with id bdf5f33c3b582f709363748d41e16c9157c104d074cc160a9923aa5fc962c220 Jul 23 00:47:03.379021 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:03.378939 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" event={"ID":"ab16a327-daf8-4f11-ad99-a97ca6d3380b","Type":"ContainerStarted","Data":"bdf5f33c3b582f709363748d41e16c9157c104d074cc160a9923aa5fc962c220"} Jul 23 00:47:04.386104 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:04.386071 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" event={"ID":"ab16a327-daf8-4f11-ad99-a97ca6d3380b","Type":"ContainerStarted","Data":"c3b04e1cc3216b3a82081846228e4d635a56c96273a3d0552969462835339c98"} Jul 23 00:47:04.386474 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:04.386182 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:04.406490 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:04.406446 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" podStartSLOduration=3.000598728 podStartE2EDuration="3.406433958s" totalImagesPullingTime="405.83523ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:47:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:47:04.404089523 +0000 UTC m=+1615.710237565" watchObservedRunningTime="2026-07-23 00:47:04.406433958 +0000 UTC m=+1615.712581951" Jul 23 00:47:10.397145 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:10.397114 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:37.404272 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.404223 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:37.404928 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.404483 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" podUID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" containerName="maas-api" containerID="cri-o://c3b04e1cc3216b3a82081846228e4d635a56c96273a3d0552969462835339c98" gracePeriod=30 Jul 23 00:47:37.564185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.564152 2575 generic.go:356] "Generic (PLEG): container finished" podID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" containerID="c3b04e1cc3216b3a82081846228e4d635a56c96273a3d0552969462835339c98" exitCode=0 Jul 23 00:47:37.564339 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.564222 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" event={"ID":"ab16a327-daf8-4f11-ad99-a97ca6d3380b","Type":"ContainerDied","Data":"c3b04e1cc3216b3a82081846228e4d635a56c96273a3d0552969462835339c98"} Jul 23 00:47:37.653127 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.653104 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:37.752703 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.752636 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") pod \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " Jul 23 00:47:37.752703 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.752697 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mmvp9\" (UniqueName: \"kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9\") pod \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " Jul 23 00:47:37.752896 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.752757 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca\") pod \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\" (UID: \"ab16a327-daf8-4f11-ad99-a97ca6d3380b\") " Jul 23 00:47:37.753127 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.753101 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "ab16a327-daf8-4f11-ad99-a97ca6d3380b" (UID: "ab16a327-daf8-4f11-ad99-a97ca6d3380b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:47:37.754666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.754630 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9" (OuterVolumeSpecName: "kube-api-access-mmvp9") pod "ab16a327-daf8-4f11-ad99-a97ca6d3380b" (UID: "ab16a327-daf8-4f11-ad99-a97ca6d3380b"). InnerVolumeSpecName "kube-api-access-mmvp9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:47:37.754666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.754655 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "ab16a327-daf8-4f11-ad99-a97ca6d3380b" (UID: "ab16a327-daf8-4f11-ad99-a97ca6d3380b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:47:37.853779 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.853748 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/ab16a327-daf8-4f11-ad99-a97ca6d3380b-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:47:37.853779 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.853773 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mmvp9\" (UniqueName: \"kubernetes.io/projected/ab16a327-daf8-4f11-ad99-a97ca6d3380b-kube-api-access-mmvp9\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:47:37.853946 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:37.853788 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/ab16a327-daf8-4f11-ad99-a97ca6d3380b-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:47:38.570700 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:38.570674 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" Jul 23 00:47:38.571103 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:38.570663 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv" event={"ID":"ab16a327-daf8-4f11-ad99-a97ca6d3380b","Type":"ContainerDied","Data":"bdf5f33c3b582f709363748d41e16c9157c104d074cc160a9923aa5fc962c220"} Jul 23 00:47:38.571103 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:38.570806 2575 scope.go:117] "RemoveContainer" containerID="c3b04e1cc3216b3a82081846228e4d635a56c96273a3d0552969462835339c98" Jul 23 00:47:38.592663 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:38.592636 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:38.596401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:38.596380 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-migrate-17c44e03-5fd4d88fd6-rrlmv"] Jul 23 00:47:39.236702 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:39.236667 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" path="/var/lib/kubelet/pods/ab16a327-daf8-4f11-ad99-a97ca6d3380b/volumes" Jul 23 00:47:45.103760 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.103729 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:47:45.104207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.104186 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" containerName="maas-api" Jul 23 00:47:45.104207 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.104204 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" containerName="maas-api" Jul 23 00:47:45.104424 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.104351 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab16a327-daf8-4f11-ad99-a97ca6d3380b" containerName="maas-api" Jul 23 00:47:45.107894 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.107872 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.110414 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.110396 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ait-c1648ae5-serving-cert\"" Jul 23 00:47:45.122923 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.122900 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:47:45.219425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.219390 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.219590 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.219433 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhkmk\" (UniqueName: \"kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.219590 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.219568 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.320382 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.320349 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nhkmk\" (UniqueName: \"kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.320567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.320446 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.320567 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.320489 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.321313 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.321287 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.323521 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.323499 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.330963 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.330943 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhkmk\" (UniqueName: \"kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk\") pod \"maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.418690 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.418630 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:45.552366 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.552344 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:47:45.554717 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:47:45.554692 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod37ce0b44_f70b_45bf_a163_bdd43793b654.slice/crio-ad62db763efcd8142007708c1b03439fca4eae3c204541ba178e05adc49875e1 WatchSource:0}: Error finding container ad62db763efcd8142007708c1b03439fca4eae3c204541ba178e05adc49875e1: Status 404 returned error can't find the container with id ad62db763efcd8142007708c1b03439fca4eae3c204541ba178e05adc49875e1 Jul 23 00:47:45.608781 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:45.608751 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" event={"ID":"37ce0b44-f70b-45bf-a163-bdd43793b654","Type":"ContainerStarted","Data":"ad62db763efcd8142007708c1b03439fca4eae3c204541ba178e05adc49875e1"} Jul 23 00:47:46.615882 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:46.615846 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" event={"ID":"37ce0b44-f70b-45bf-a163-bdd43793b654","Type":"ContainerStarted","Data":"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6"} Jul 23 00:47:46.616240 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:46.615958 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:47:46.634680 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:46.634630 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" podStartSLOduration=1.216564829 podStartE2EDuration="1.63461653s" totalImagesPullingTime="418.051701ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:47:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:47:46.632551807 +0000 UTC m=+1657.938699826" watchObservedRunningTime="2026-07-23 00:47:46.63461653 +0000 UTC m=+1657.940764523" Jul 23 00:47:52.626276 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:47:52.626242 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:48:21.371332 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.371299 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:48:21.371880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.371547 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" podUID="37ce0b44-f70b-45bf-a163-bdd43793b654" containerName="maas-api" containerID="cri-o://cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6" gracePeriod=30 Jul 23 00:48:21.620028 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.620004 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:48:21.630616 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.630557 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca\") pod \"37ce0b44-f70b-45bf-a163-bdd43793b654\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " Jul 23 00:48:21.630616 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.630601 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nhkmk\" (UniqueName: \"kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk\") pod \"37ce0b44-f70b-45bf-a163-bdd43793b654\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " Jul 23 00:48:21.630794 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.630639 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls\") pod \"37ce0b44-f70b-45bf-a163-bdd43793b654\" (UID: \"37ce0b44-f70b-45bf-a163-bdd43793b654\") " Jul 23 00:48:21.630941 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.630912 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "37ce0b44-f70b-45bf-a163-bdd43793b654" (UID: "37ce0b44-f70b-45bf-a163-bdd43793b654"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:48:21.632556 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.632528 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk" (OuterVolumeSpecName: "kube-api-access-nhkmk") pod "37ce0b44-f70b-45bf-a163-bdd43793b654" (UID: "37ce0b44-f70b-45bf-a163-bdd43793b654"). InnerVolumeSpecName "kube-api-access-nhkmk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:48:21.632696 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.632663 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "37ce0b44-f70b-45bf-a163-bdd43793b654" (UID: "37ce0b44-f70b-45bf-a163-bdd43793b654"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:48:21.732335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.732301 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/37ce0b44-f70b-45bf-a163-bdd43793b654-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:48:21.732335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.732331 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nhkmk\" (UniqueName: \"kubernetes.io/projected/37ce0b44-f70b-45bf-a163-bdd43793b654-kube-api-access-nhkmk\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:48:21.732335 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.732341 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/37ce0b44-f70b-45bf-a163-bdd43793b654-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:48:21.802898 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.802867 2575 generic.go:356] "Generic (PLEG): container finished" podID="37ce0b44-f70b-45bf-a163-bdd43793b654" containerID="cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6" exitCode=0 Jul 23 00:48:21.803032 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.802920 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" Jul 23 00:48:21.803032 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.802952 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" event={"ID":"37ce0b44-f70b-45bf-a163-bdd43793b654","Type":"ContainerDied","Data":"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6"} Jul 23 00:48:21.803032 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.803001 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb" event={"ID":"37ce0b44-f70b-45bf-a163-bdd43793b654","Type":"ContainerDied","Data":"ad62db763efcd8142007708c1b03439fca4eae3c204541ba178e05adc49875e1"} Jul 23 00:48:21.803032 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.803024 2575 scope.go:117] "RemoveContainer" containerID="cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6" Jul 23 00:48:21.812297 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.812279 2575 scope.go:117] "RemoveContainer" containerID="cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6" Jul 23 00:48:21.812535 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:48:21.812514 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6\": container with ID starting with cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6 not found: ID does not exist" containerID="cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6" Jul 23 00:48:21.812597 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.812543 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6"} err="failed to get container status \"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6\": rpc error: code = NotFound desc = could not find container \"cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6\": container with ID starting with cfd31080a73ab020e8f0c97b8ae976eef8c0aa9cd5aa5b81066960b3960edcc6 not found: ID does not exist" Jul 23 00:48:21.825998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.825972 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:48:21.830176 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:21.830153 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ait-c1648ae5-6ffbc964b7-tzzhb"] Jul 23 00:48:23.236027 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:23.235997 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37ce0b44-f70b-45bf-a163-bdd43793b654" path="/var/lib/kubelet/pods/37ce0b44-f70b-45bf-a163-bdd43793b654/volumes" Jul 23 00:48:30.383177 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.383144 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:48:30.383615 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.383562 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37ce0b44-f70b-45bf-a163-bdd43793b654" containerName="maas-api" Jul 23 00:48:30.383615 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.383574 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="37ce0b44-f70b-45bf-a163-bdd43793b654" containerName="maas-api" Jul 23 00:48:30.383689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.383632 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="37ce0b44-f70b-45bf-a163-bdd43793b654" containerName="maas-api" Jul 23 00:48:30.387953 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.387936 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.390584 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.390558 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-derive-627b9dcc-serving-cert\"" Jul 23 00:48:30.397576 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.397554 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:48:30.511561 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.511535 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.511689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.511566 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7fws\" (UniqueName: \"kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.511689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.511663 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.612240 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.612206 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.612386 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.612298 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.612386 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.612324 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7fws\" (UniqueName: \"kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.612822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.612790 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.614739 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.614705 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.620441 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.620420 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7fws\" (UniqueName: \"kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws\") pod \"maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.700272 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.700197 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:30.830561 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.830533 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:48:30.832147 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:48:30.832113 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5dd0d5_57b4_4b87_bf33_23ccad244d23.slice/crio-f2164a862d6545c667f5ba1c6ba10f382f0aa7b4067fca3dfed33b14b5013b31 WatchSource:0}: Error finding container f2164a862d6545c667f5ba1c6ba10f382f0aa7b4067fca3dfed33b14b5013b31: Status 404 returned error can't find the container with id f2164a862d6545c667f5ba1c6ba10f382f0aa7b4067fca3dfed33b14b5013b31 Jul 23 00:48:30.852951 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:30.852919 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" event={"ID":"da5dd0d5-57b4-4b87-bf33-23ccad244d23","Type":"ContainerStarted","Data":"f2164a862d6545c667f5ba1c6ba10f382f0aa7b4067fca3dfed33b14b5013b31"} Jul 23 00:48:31.859137 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:31.859099 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" event={"ID":"da5dd0d5-57b4-4b87-bf33-23ccad244d23","Type":"ContainerStarted","Data":"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc"} Jul 23 00:48:31.859547 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:31.859209 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:48:31.877332 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:31.877286 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" podStartSLOduration=1.401463654 podStartE2EDuration="1.877272756s" totalImagesPullingTime="475.809102ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:48:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:48:31.875194194 +0000 UTC m=+1703.181342189" watchObservedRunningTime="2026-07-23 00:48:31.877272756 +0000 UTC m=+1703.183420751" Jul 23 00:48:37.869943 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:48:37.869914 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:49:05.653019 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:05.652980 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:49:05.653611 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:05.653200 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" podUID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" containerName="maas-api" containerID="cri-o://bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc" gracePeriod=30 Jul 23 00:49:05.897377 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:05.897353 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:49:06.002372 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.002301 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls\") pod \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " Jul 23 00:49:06.002499 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.002424 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7fws\" (UniqueName: \"kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws\") pod \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " Jul 23 00:49:06.002499 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.002452 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca\") pod \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\" (UID: \"da5dd0d5-57b4-4b87-bf33-23ccad244d23\") " Jul 23 00:49:06.002890 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.002863 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "da5dd0d5-57b4-4b87-bf33-23ccad244d23" (UID: "da5dd0d5-57b4-4b87-bf33-23ccad244d23"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:49:06.004470 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.004442 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws" (OuterVolumeSpecName: "kube-api-access-b7fws") pod "da5dd0d5-57b4-4b87-bf33-23ccad244d23" (UID: "da5dd0d5-57b4-4b87-bf33-23ccad244d23"). InnerVolumeSpecName "kube-api-access-b7fws". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:49:06.004470 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.004453 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "da5dd0d5-57b4-4b87-bf33-23ccad244d23" (UID: "da5dd0d5-57b4-4b87-bf33-23ccad244d23"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:49:06.038493 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.038461 2575 generic.go:356] "Generic (PLEG): container finished" podID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" containerID="bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc" exitCode=0 Jul 23 00:49:06.038613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.038508 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" Jul 23 00:49:06.038613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.038536 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" event={"ID":"da5dd0d5-57b4-4b87-bf33-23ccad244d23","Type":"ContainerDied","Data":"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc"} Jul 23 00:49:06.038613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.038582 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr" event={"ID":"da5dd0d5-57b4-4b87-bf33-23ccad244d23","Type":"ContainerDied","Data":"f2164a862d6545c667f5ba1c6ba10f382f0aa7b4067fca3dfed33b14b5013b31"} Jul 23 00:49:06.038613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.038607 2575 scope.go:117] "RemoveContainer" containerID="bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc" Jul 23 00:49:06.048123 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.048104 2575 scope.go:117] "RemoveContainer" containerID="bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc" Jul 23 00:49:06.048360 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:49:06.048340 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc\": container with ID starting with bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc not found: ID does not exist" containerID="bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc" Jul 23 00:49:06.048453 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.048369 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc"} err="failed to get container status \"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc\": rpc error: code = NotFound desc = could not find container \"bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc\": container with ID starting with bad75decc7b5d1cc57f4157a6edb2fe3862cb1f3b59137159293fad2372bb3dc not found: ID does not exist" Jul 23 00:49:06.061446 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.061424 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:49:06.063555 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.063537 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-derive-627b9dcc-bdbbf98d7-kd9zr"] Jul 23 00:49:06.103291 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.103271 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7fws\" (UniqueName: \"kubernetes.io/projected/da5dd0d5-57b4-4b87-bf33-23ccad244d23-kube-api-access-b7fws\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:49:06.103383 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.103293 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/da5dd0d5-57b4-4b87-bf33-23ccad244d23-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:49:06.103383 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:06.103306 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/da5dd0d5-57b4-4b87-bf33-23ccad244d23-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:49:07.236253 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:49:07.236223 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" path="/var/lib/kubelet/pods/da5dd0d5-57b4-4b87-bf33-23ccad244d23/volumes" Jul 23 00:50:09.252524 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:09.252427 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:50:09.259311 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:09.257418 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:50:50.950652 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.950607 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:50:50.951731 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.951703 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" containerName="maas-api" Jul 23 00:50:50.951731 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.951732 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" containerName="maas-api" Jul 23 00:50:50.952109 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.952091 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="da5dd0d5-57b4-4b87-bf33-23ccad244d23" containerName="maas-api" Jul 23 00:50:50.956675 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.956641 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:50.959880 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.959844 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-89b85334-serving-cert\"" Jul 23 00:50:50.963417 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:50.963394 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:50:51.078776 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.078740 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.078776 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.078777 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ws45\" (UniqueName: \"kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.078975 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.078910 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.179608 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.179575 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.179764 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.179635 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.179764 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.179747 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ws45\" (UniqueName: \"kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.180216 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.180197 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.181998 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.181973 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.187886 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.187868 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ws45\" (UniqueName: \"kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45\") pod \"maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.269078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.269014 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:51.612598 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:51.612569 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:50:51.614165 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:50:51.614131 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50033ba7_ff0e_4bd1_a369_4f767b86790e.slice/crio-47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f WatchSource:0}: Error finding container 47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f: Status 404 returned error can't find the container with id 47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f Jul 23 00:50:52.611562 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:52.611529 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" event={"ID":"50033ba7-ff0e-4bd1-a369-4f767b86790e","Type":"ContainerStarted","Data":"47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f"} Jul 23 00:50:53.618572 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:53.618477 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" event={"ID":"50033ba7-ff0e-4bd1-a369-4f767b86790e","Type":"ContainerStarted","Data":"a265f21163754d2c2a128bcb6b8ca74bf12a32c2d2e8fd432bb8c0be71dd755c"} Jul 23 00:50:53.619016 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:53.618596 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:50:53.636521 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:53.636462 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" podStartSLOduration=1.894256642 podStartE2EDuration="3.636448179s" totalImagesPullingTime="1.742191537s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:50:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:50:53.635643912 +0000 UTC m=+1844.941791921" watchObservedRunningTime="2026-07-23 00:50:53.636448179 +0000 UTC m=+1844.942596172" Jul 23 00:50:59.630191 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:50:59.630164 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:51:13.475726 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.475690 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:51:13.476221 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.475952 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" podUID="50033ba7-ff0e-4bd1-a369-4f767b86790e" containerName="maas-api" containerID="cri-o://a265f21163754d2c2a128bcb6b8ca74bf12a32c2d2e8fd432bb8c0be71dd755c" gracePeriod=30 Jul 23 00:51:13.725642 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.725576 2575 generic.go:356] "Generic (PLEG): container finished" podID="50033ba7-ff0e-4bd1-a369-4f767b86790e" containerID="a265f21163754d2c2a128bcb6b8ca74bf12a32c2d2e8fd432bb8c0be71dd755c" exitCode=0 Jul 23 00:51:13.725779 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.725648 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" event={"ID":"50033ba7-ff0e-4bd1-a369-4f767b86790e","Type":"ContainerDied","Data":"a265f21163754d2c2a128bcb6b8ca74bf12a32c2d2e8fd432bb8c0be71dd755c"} Jul 23 00:51:13.725779 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.725697 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" event={"ID":"50033ba7-ff0e-4bd1-a369-4f767b86790e","Type":"ContainerDied","Data":"47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f"} Jul 23 00:51:13.725779 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.725717 2575 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="47ecf74626d0e89b8c4ef08a502ef0243e052ae32a93c2980372f38b5111e96f" Jul 23 00:51:13.726809 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.726791 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:51:13.776328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.776297 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca\") pod \"50033ba7-ff0e-4bd1-a369-4f767b86790e\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " Jul 23 00:51:13.776328 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.776328 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4ws45\" (UniqueName: \"kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45\") pod \"50033ba7-ff0e-4bd1-a369-4f767b86790e\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " Jul 23 00:51:13.776523 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.776374 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls\") pod \"50033ba7-ff0e-4bd1-a369-4f767b86790e\" (UID: \"50033ba7-ff0e-4bd1-a369-4f767b86790e\") " Jul 23 00:51:13.776689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.776662 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "50033ba7-ff0e-4bd1-a369-4f767b86790e" (UID: "50033ba7-ff0e-4bd1-a369-4f767b86790e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:51:13.778371 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.778343 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45" (OuterVolumeSpecName: "kube-api-access-4ws45") pod "50033ba7-ff0e-4bd1-a369-4f767b86790e" (UID: "50033ba7-ff0e-4bd1-a369-4f767b86790e"). InnerVolumeSpecName "kube-api-access-4ws45". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:51:13.778477 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.778407 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "50033ba7-ff0e-4bd1-a369-4f767b86790e" (UID: "50033ba7-ff0e-4bd1-a369-4f767b86790e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:51:13.878035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.878012 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/50033ba7-ff0e-4bd1-a369-4f767b86790e-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:51:13.878035 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.878034 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/50033ba7-ff0e-4bd1-a369-4f767b86790e-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:51:13.878169 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:13.878045 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4ws45\" (UniqueName: \"kubernetes.io/projected/50033ba7-ff0e-4bd1-a369-4f767b86790e-kube-api-access-4ws45\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:51:14.731073 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:14.731043 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76" Jul 23 00:51:14.753008 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:14.752981 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:51:14.757795 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:14.757770 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-89b85334-7dc9d7bf69-6cz76"] Jul 23 00:51:15.236647 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:15.236611 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50033ba7-ff0e-4bd1-a369-4f767b86790e" path="/var/lib/kubelet/pods/50033ba7-ff0e-4bd1-a369-4f767b86790e/volumes" Jul 23 00:51:47.135565 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.135483 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:51:47.136083 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.135946 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50033ba7-ff0e-4bd1-a369-4f767b86790e" containerName="maas-api" Jul 23 00:51:47.136083 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.135961 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="50033ba7-ff0e-4bd1-a369-4f767b86790e" containerName="maas-api" Jul 23 00:51:47.136083 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.136026 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="50033ba7-ff0e-4bd1-a369-4f767b86790e" containerName="maas-api" Jul 23 00:51:47.138981 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.138959 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.142150 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.142125 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-mt-3c46ebab-serving-cert\"" Jul 23 00:51:47.153013 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.152987 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:51:47.169416 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.169391 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.169529 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.169449 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.169529 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.169468 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlmt8\" (UniqueName: \"kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.270666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.270637 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.270793 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.270672 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wlmt8\" (UniqueName: \"kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.270793 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:51:47.270757 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-serving-cert: secret "maas-api-e2e-mt-3c46ebab-serving-cert" not found Jul 23 00:51:47.270942 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:51:47.270823 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls podName:88c2e790-9d59-45c1-a1b1-134310d84478 nodeName:}" failed. No retries permitted until 2026-07-23 00:51:47.770801228 +0000 UTC m=+1899.076949204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls") pod "maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" (UID: "88c2e790-9d59-45c1-a1b1-134310d84478") : secret "maas-api-e2e-mt-3c46ebab-serving-cert" not found Jul 23 00:51:47.271132 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.271112 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.271770 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.271750 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.281155 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.281137 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlmt8\" (UniqueName: \"kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.775491 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.775451 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:47.778025 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:47.778004 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") pod \"maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:48.051505 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.051426 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:48.391947 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.391916 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:51:48.393298 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:51:48.393274 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c2e790_9d59_45c1_a1b1_134310d84478.slice/crio-f4ca900c4b3c1b989bcf91669964700402739b5a5fd3b088175a808dadfcedb2 WatchSource:0}: Error finding container f4ca900c4b3c1b989bcf91669964700402739b5a5fd3b088175a808dadfcedb2: Status 404 returned error can't find the container with id f4ca900c4b3c1b989bcf91669964700402739b5a5fd3b088175a808dadfcedb2 Jul 23 00:51:48.395181 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.395156 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:51:48.917158 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.917126 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" event={"ID":"88c2e790-9d59-45c1-a1b1-134310d84478","Type":"ContainerStarted","Data":"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515"} Jul 23 00:51:48.917330 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.917178 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" event={"ID":"88c2e790-9d59-45c1-a1b1-134310d84478","Type":"ContainerStarted","Data":"f4ca900c4b3c1b989bcf91669964700402739b5a5fd3b088175a808dadfcedb2"} Jul 23 00:51:48.917330 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.917275 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:51:48.934788 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:48.934740 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" podStartSLOduration=1.6068163709999999 podStartE2EDuration="1.934722877s" totalImagesPullingTime="327.906506ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:51:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:51:48.933776117 +0000 UTC m=+1900.239924111" watchObservedRunningTime="2026-07-23 00:51:48.934722877 +0000 UTC m=+1900.240870870" Jul 23 00:51:54.927924 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:51:54.927894 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:52:45.770648 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:45.770614 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:52:45.771576 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:45.771539 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" podUID="88c2e790-9d59-45c1-a1b1-134310d84478" containerName="maas-api" containerID="cri-o://73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515" gracePeriod=30 Jul 23 00:52:46.027992 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.027935 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:52:46.161483 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.161452 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wlmt8\" (UniqueName: \"kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8\") pod \"88c2e790-9d59-45c1-a1b1-134310d84478\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " Jul 23 00:52:46.161625 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.161490 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca\") pod \"88c2e790-9d59-45c1-a1b1-134310d84478\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " Jul 23 00:52:46.161625 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.161531 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") pod \"88c2e790-9d59-45c1-a1b1-134310d84478\" (UID: \"88c2e790-9d59-45c1-a1b1-134310d84478\") " Jul 23 00:52:46.161930 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.161897 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "88c2e790-9d59-45c1-a1b1-134310d84478" (UID: "88c2e790-9d59-45c1-a1b1-134310d84478"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:52:46.163613 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.163588 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8" (OuterVolumeSpecName: "kube-api-access-wlmt8") pod "88c2e790-9d59-45c1-a1b1-134310d84478" (UID: "88c2e790-9d59-45c1-a1b1-134310d84478"). InnerVolumeSpecName "kube-api-access-wlmt8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:52:46.163711 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.163625 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "88c2e790-9d59-45c1-a1b1-134310d84478" (UID: "88c2e790-9d59-45c1-a1b1-134310d84478"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:52:46.209408 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.209378 2575 generic.go:356] "Generic (PLEG): container finished" podID="88c2e790-9d59-45c1-a1b1-134310d84478" containerID="73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515" exitCode=0 Jul 23 00:52:46.209510 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.209423 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" Jul 23 00:52:46.209510 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.209441 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" event={"ID":"88c2e790-9d59-45c1-a1b1-134310d84478","Type":"ContainerDied","Data":"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515"} Jul 23 00:52:46.209510 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.209494 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg" event={"ID":"88c2e790-9d59-45c1-a1b1-134310d84478","Type":"ContainerDied","Data":"f4ca900c4b3c1b989bcf91669964700402739b5a5fd3b088175a808dadfcedb2"} Jul 23 00:52:46.209672 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.209524 2575 scope.go:117] "RemoveContainer" containerID="73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515" Jul 23 00:52:46.221451 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.221431 2575 scope.go:117] "RemoveContainer" containerID="73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515" Jul 23 00:52:46.221913 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:52:46.221885 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515\": container with ID starting with 73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515 not found: ID does not exist" containerID="73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515" Jul 23 00:52:46.222014 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.221922 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515"} err="failed to get container status \"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515\": rpc error: code = NotFound desc = could not find container \"73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515\": container with ID starting with 73ffde3c725a0a38a7337ce085652b637cac05877f85bf9e780b5b395a596515 not found: ID does not exist" Jul 23 00:52:46.231138 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.231115 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:52:46.234763 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.234741 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-mt-3c46ebab-5ddf9ffccf-t55pg"] Jul 23 00:52:46.262729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.262708 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wlmt8\" (UniqueName: \"kubernetes.io/projected/88c2e790-9d59-45c1-a1b1-134310d84478-kube-api-access-wlmt8\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:52:46.262729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.262728 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/88c2e790-9d59-45c1-a1b1-134310d84478-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:52:46.262844 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:46.262741 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/88c2e790-9d59-45c1-a1b1-134310d84478-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:52:47.237017 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:52:47.236984 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88c2e790-9d59-45c1-a1b1-134310d84478" path="/var/lib/kubelet/pods/88c2e790-9d59-45c1-a1b1-134310d84478/volumes" Jul 23 00:54:40.066527 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.066439 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:54:40.067058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.066884 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88c2e790-9d59-45c1-a1b1-134310d84478" containerName="maas-api" Jul 23 00:54:40.067058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.066899 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="88c2e790-9d59-45c1-a1b1-134310d84478" containerName="maas-api" Jul 23 00:54:40.067058 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.066975 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="88c2e790-9d59-45c1-a1b1-134310d84478" containerName="maas-api" Jul 23 00:54:40.069972 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.069952 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.072369 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.072347 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-551ea4-serving-cert\"" Jul 23 00:54:40.081601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.081578 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:54:40.119441 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.119413 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.119550 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.119453 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zdjl9\" (UniqueName: \"kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.119550 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.119528 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.220049 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.220018 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.220202 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.220062 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zdjl9\" (UniqueName: \"kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.220202 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.220104 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.220202 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:54:40.220159 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-serving-cert: secret "maas-api-e2e-inf-a-551ea4-serving-cert" not found Jul 23 00:54:40.220304 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:54:40.220220 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls podName:28a35f3e-53e6-4bbc-895e-a718dc1f053b nodeName:}" failed. No retries permitted until 2026-07-23 00:54:40.720203372 +0000 UTC m=+2072.026351345 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls") pod "maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" (UID: "28a35f3e-53e6-4bbc-895e-a718dc1f053b") : secret "maas-api-e2e-inf-a-551ea4-serving-cert" not found Jul 23 00:54:40.220653 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.220637 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.230474 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.230451 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zdjl9\" (UniqueName: \"kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.724030 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.724000 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.726369 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.726348 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") pod \"maas-api-e2e-inf-a-551ea4-6664456876-5pz4d\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:40.981664 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:40.981577 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:41.116979 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:41.116953 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:54:41.119087 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:54:41.119060 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28a35f3e_53e6_4bbc_895e_a718dc1f053b.slice/crio-60b03829a61996fbb327fdc5710d8e56af24bb4e4967603f6209931eba5baf60 WatchSource:0}: Error finding container 60b03829a61996fbb327fdc5710d8e56af24bb4e4967603f6209931eba5baf60: Status 404 returned error can't find the container with id 60b03829a61996fbb327fdc5710d8e56af24bb4e4967603f6209931eba5baf60 Jul 23 00:54:41.806992 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:41.806946 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" event={"ID":"28a35f3e-53e6-4bbc-895e-a718dc1f053b","Type":"ContainerStarted","Data":"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005"} Jul 23 00:54:41.807173 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:41.807021 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:41.807173 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:41.807047 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" event={"ID":"28a35f3e-53e6-4bbc-895e-a718dc1f053b","Type":"ContainerStarted","Data":"60b03829a61996fbb327fdc5710d8e56af24bb4e4967603f6209931eba5baf60"} Jul 23 00:54:41.828269 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:41.828173 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" podStartSLOduration=1.389910311 podStartE2EDuration="1.828157744s" totalImagesPullingTime="438.247433ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:54:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:54:41.826374397 +0000 UTC m=+2073.132522391" watchObservedRunningTime="2026-07-23 00:54:41.828157744 +0000 UTC m=+2073.134305738" Jul 23 00:54:47.819568 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:47.819538 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:54:48.112043 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.111950 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:54:48.115575 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.115557 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.118064 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.118044 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-fb82e4-serving-cert\"" Jul 23 00:54:48.126555 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.126529 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:54:48.184563 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.184537 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.184697 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.184606 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvm5\" (UniqueName: \"kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.184745 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.184688 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.285098 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.285068 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvm5\" (UniqueName: \"kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.285264 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.285118 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.285264 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.285151 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.285383 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:54:48.285316 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-serving-cert: secret "maas-api-e2e-inf-b-fb82e4-serving-cert" not found Jul 23 00:54:48.285444 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:54:48.285387 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls podName:6d8b2f5f-eaca-41ab-bd18-386350c74b1e nodeName:}" failed. No retries permitted until 2026-07-23 00:54:48.785371441 +0000 UTC m=+2080.091519413 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls") pod "maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" (UID: "6d8b2f5f-eaca-41ab-bd18-386350c74b1e") : secret "maas-api-e2e-inf-b-fb82e4-serving-cert" not found Jul 23 00:54:48.285733 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.285715 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.294465 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.294438 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvm5\" (UniqueName: \"kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.789648 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.789609 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:48.792047 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:48.792019 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") pod \"maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:49.026689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.026658 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:49.369822 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.369792 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:54:49.371669 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:54:49.371638 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d8b2f5f_eaca_41ab_bd18_386350c74b1e.slice/crio-1ab369ffdb27141085cfaab3efde473c5ea0baf6a83eb64fbe2640e8168abfb4 WatchSource:0}: Error finding container 1ab369ffdb27141085cfaab3efde473c5ea0baf6a83eb64fbe2640e8168abfb4: Status 404 returned error can't find the container with id 1ab369ffdb27141085cfaab3efde473c5ea0baf6a83eb64fbe2640e8168abfb4 Jul 23 00:54:49.855431 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.855392 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" event={"ID":"6d8b2f5f-eaca-41ab-bd18-386350c74b1e","Type":"ContainerStarted","Data":"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4"} Jul 23 00:54:49.855604 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.855461 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:54:49.855604 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.855482 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" event={"ID":"6d8b2f5f-eaca-41ab-bd18-386350c74b1e","Type":"ContainerStarted","Data":"1ab369ffdb27141085cfaab3efde473c5ea0baf6a83eb64fbe2640e8168abfb4"} Jul 23 00:54:49.871908 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:49.871824 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" podStartSLOduration=1.542441528 podStartE2EDuration="1.871807571s" totalImagesPullingTime="329.366043ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:54:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:54:49.871090229 +0000 UTC m=+2081.177238223" watchObservedRunningTime="2026-07-23 00:54:49.871807571 +0000 UTC m=+2081.177955565" Jul 23 00:54:55.866028 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:54:55.865997 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:55:09.287890 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:55:09.287759 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:55:09.300051 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:55:09.294465 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 00:56:34.807982 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:34.807945 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:56:34.808521 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:34.808215 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" podUID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" containerName="maas-api" containerID="cri-o://e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005" gracePeriod=30 Jul 23 00:56:35.057206 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.057179 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:56:35.074406 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.074355 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") pod \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " Jul 23 00:56:35.074406 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.074389 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca\") pod \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " Jul 23 00:56:35.074537 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.074435 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zdjl9\" (UniqueName: \"kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9\") pod \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\" (UID: \"28a35f3e-53e6-4bbc-895e-a718dc1f053b\") " Jul 23 00:56:35.074777 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.074753 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "28a35f3e-53e6-4bbc-895e-a718dc1f053b" (UID: "28a35f3e-53e6-4bbc-895e-a718dc1f053b"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:56:35.076379 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.076355 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "28a35f3e-53e6-4bbc-895e-a718dc1f053b" (UID: "28a35f3e-53e6-4bbc-895e-a718dc1f053b"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:56:35.076506 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.076485 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9" (OuterVolumeSpecName: "kube-api-access-zdjl9") pod "28a35f3e-53e6-4bbc-895e-a718dc1f053b" (UID: "28a35f3e-53e6-4bbc-895e-a718dc1f053b"). InnerVolumeSpecName "kube-api-access-zdjl9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:56:35.175425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.175397 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/28a35f3e-53e6-4bbc-895e-a718dc1f053b-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:35.175425 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.175420 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/28a35f3e-53e6-4bbc-895e-a718dc1f053b-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:35.175575 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.175430 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zdjl9\" (UniqueName: \"kubernetes.io/projected/28a35f3e-53e6-4bbc-895e-a718dc1f053b-kube-api-access-zdjl9\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:35.426758 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.426682 2575 generic.go:356] "Generic (PLEG): container finished" podID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" containerID="e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005" exitCode=0 Jul 23 00:56:35.426758 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.426723 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" Jul 23 00:56:35.426948 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.426760 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" event={"ID":"28a35f3e-53e6-4bbc-895e-a718dc1f053b","Type":"ContainerDied","Data":"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005"} Jul 23 00:56:35.426948 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.426815 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d" event={"ID":"28a35f3e-53e6-4bbc-895e-a718dc1f053b","Type":"ContainerDied","Data":"60b03829a61996fbb327fdc5710d8e56af24bb4e4967603f6209931eba5baf60"} Jul 23 00:56:35.426948 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.426858 2575 scope.go:117] "RemoveContainer" containerID="e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005" Jul 23 00:56:35.436165 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.436145 2575 scope.go:117] "RemoveContainer" containerID="e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005" Jul 23 00:56:35.436414 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:56:35.436395 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005\": container with ID starting with e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005 not found: ID does not exist" containerID="e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005" Jul 23 00:56:35.436459 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.436421 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005"} err="failed to get container status \"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005\": rpc error: code = NotFound desc = could not find container \"e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005\": container with ID starting with e305e068535707f39afc3dc8cd10e5c2080ff119ec40e61faf2f4080a15b2005 not found: ID does not exist" Jul 23 00:56:35.445396 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.445374 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:56:35.447559 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:35.447539 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-551ea4-6664456876-5pz4d"] Jul 23 00:56:37.236506 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:37.236470 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" path="/var/lib/kubelet/pods/28a35f3e-53e6-4bbc-895e-a718dc1f053b/volumes" Jul 23 00:56:56.988554 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:56.988521 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:56:56.989113 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:56.988709 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" podUID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" containerName="maas-api" containerID="cri-o://ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4" gracePeriod=30 Jul 23 00:56:57.241805 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.241741 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:56:57.379503 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.379475 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") pod \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " Jul 23 00:56:57.379688 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.379509 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca\") pod \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " Jul 23 00:56:57.379688 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.379539 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tpvm5\" (UniqueName: \"kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5\") pod \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\" (UID: \"6d8b2f5f-eaca-41ab-bd18-386350c74b1e\") " Jul 23 00:56:57.379928 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.379904 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6d8b2f5f-eaca-41ab-bd18-386350c74b1e" (UID: "6d8b2f5f-eaca-41ab-bd18-386350c74b1e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:56:57.381689 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.381665 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5" (OuterVolumeSpecName: "kube-api-access-tpvm5") pod "6d8b2f5f-eaca-41ab-bd18-386350c74b1e" (UID: "6d8b2f5f-eaca-41ab-bd18-386350c74b1e"). InnerVolumeSpecName "kube-api-access-tpvm5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:56:57.381777 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.381689 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6d8b2f5f-eaca-41ab-bd18-386350c74b1e" (UID: "6d8b2f5f-eaca-41ab-bd18-386350c74b1e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:56:57.481045 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.481021 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:57.481045 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.481043 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:57.481164 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.481053 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tpvm5\" (UniqueName: \"kubernetes.io/projected/6d8b2f5f-eaca-41ab-bd18-386350c74b1e-kube-api-access-tpvm5\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:56:57.547797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.547734 2575 generic.go:356] "Generic (PLEG): container finished" podID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" containerID="ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4" exitCode=0 Jul 23 00:56:57.547797 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.547781 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" Jul 23 00:56:57.547934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.547819 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" event={"ID":"6d8b2f5f-eaca-41ab-bd18-386350c74b1e","Type":"ContainerDied","Data":"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4"} Jul 23 00:56:57.547934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.547893 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck" event={"ID":"6d8b2f5f-eaca-41ab-bd18-386350c74b1e","Type":"ContainerDied","Data":"1ab369ffdb27141085cfaab3efde473c5ea0baf6a83eb64fbe2640e8168abfb4"} Jul 23 00:56:57.547934 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.547916 2575 scope.go:117] "RemoveContainer" containerID="ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4" Jul 23 00:56:57.557043 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.557026 2575 scope.go:117] "RemoveContainer" containerID="ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4" Jul 23 00:56:57.557289 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:56:57.557272 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4\": container with ID starting with ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4 not found: ID does not exist" containerID="ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4" Jul 23 00:56:57.557332 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.557297 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4"} err="failed to get container status \"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4\": rpc error: code = NotFound desc = could not find container \"ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4\": container with ID starting with ed0e297797638271a44962e382aa9c84813e8f7c955236fa29364b518fb00fd4 not found: ID does not exist" Jul 23 00:56:57.570293 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.570267 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:56:57.573691 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:57.573667 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-fb82e4-86d6545f77-7frck"] Jul 23 00:56:59.236772 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:56:59.236733 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" path="/var/lib/kubelet/pods/6d8b2f5f-eaca-41ab-bd18-386350c74b1e/volumes" Jul 23 00:57:09.353186 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:09.353150 2575 scope.go:117] "RemoveContainer" containerID="a265f21163754d2c2a128bcb6b8ca74bf12a32c2d2e8fd432bb8c0be71dd755c" Jul 23 00:57:11.948081 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948037 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948406 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" containerName="maas-api" Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948418 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" containerName="maas-api" Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948434 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" containerName="maas-api" Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948439 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" containerName="maas-api" Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948504 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="28a35f3e-53e6-4bbc-895e-a718dc1f053b" containerName="maas-api" Jul 23 00:57:11.948552 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.948512 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="6d8b2f5f-eaca-41ab-bd18-386350c74b1e" containerName="maas-api" Jul 23 00:57:11.953092 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.953069 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:11.955935 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.955914 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-a-23cf83-serving-cert\"" Jul 23 00:57:11.965274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:11.965250 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:12.104321 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.104284 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.104503 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.104347 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zls56\" (UniqueName: \"kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.104503 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.104405 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.205601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.205517 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zls56\" (UniqueName: \"kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.205601 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.205561 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.205857 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.205610 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.205857 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:57:12.205722 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-serving-cert: secret "maas-api-e2e-api-a-23cf83-serving-cert" not found Jul 23 00:57:12.205857 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:57:12.205789 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls podName:4c4d063b-5d27-406b-af3b-c2fcc90677ee nodeName:}" failed. No retries permitted until 2026-07-23 00:57:12.70577339 +0000 UTC m=+2224.011921362 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls") pod "maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" (UID: "4c4d063b-5d27-406b-af3b-c2fcc90677ee") : secret "maas-api-e2e-api-a-23cf83-serving-cert" not found Jul 23 00:57:12.206216 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.206198 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.214729 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.214704 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zls56\" (UniqueName: \"kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.709918 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.709884 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.712239 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.712217 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") pod \"maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:12.864340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:12.864300 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:13.004078 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:13.003954 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:13.007375 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:57:13.007340 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c4d063b_5d27_406b_af3b_c2fcc90677ee.slice/crio-771e63222202390f14303ead85ae032e53e599f55da196b5c18f503741d671a0 WatchSource:0}: Error finding container 771e63222202390f14303ead85ae032e53e599f55da196b5c18f503741d671a0: Status 404 returned error can't find the container with id 771e63222202390f14303ead85ae032e53e599f55da196b5c18f503741d671a0 Jul 23 00:57:13.008722 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:13.008701 2575 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 23 00:57:13.634019 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:13.633942 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" event={"ID":"4c4d063b-5d27-406b-af3b-c2fcc90677ee","Type":"ContainerStarted","Data":"771e63222202390f14303ead85ae032e53e599f55da196b5c18f503741d671a0"} Jul 23 00:57:14.640464 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:14.640433 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" event={"ID":"4c4d063b-5d27-406b-af3b-c2fcc90677ee","Type":"ContainerStarted","Data":"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c"} Jul 23 00:57:14.640890 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:14.640499 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:14.660241 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:14.660183 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" podStartSLOduration=3.034844782 podStartE2EDuration="3.660163826s" totalImagesPullingTime="625.319044ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:57:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:57:14.657072859 +0000 UTC m=+2225.963220853" watchObservedRunningTime="2026-07-23 00:57:14.660163826 +0000 UTC m=+2225.966311822" Jul 23 00:57:19.985340 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:19.985300 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:57:19.989182 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:19.989162 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:19.992690 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:19.992668 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-api-b-e6fe2f-serving-cert\"" Jul 23 00:57:20.007888 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.007864 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:57:20.076666 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.076636 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.076789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.076689 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xntnm\" (UniqueName: \"kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.076886 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.076823 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.177711 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.177682 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.177913 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.177728 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xntnm\" (UniqueName: \"kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.177913 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.177777 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.177913 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:57:20.177851 2575 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-serving-cert: secret "maas-api-e2e-api-b-e6fe2f-serving-cert" not found Jul 23 00:57:20.177913 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:57:20.177912 2575 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls podName:8fce541a-8019-415b-9f84-88a7b12a3c00 nodeName:}" failed. No retries permitted until 2026-07-23 00:57:20.67789728 +0000 UTC m=+2231.984045253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls") pod "maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" (UID: "8fce541a-8019-415b-9f84-88a7b12a3c00") : secret "maas-api-e2e-api-b-e6fe2f-serving-cert" not found Jul 23 00:57:20.178342 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.178324 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.198650 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.198633 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xntnm\" (UniqueName: \"kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.651973 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.651947 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:20.682936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.682904 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.685735 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.685710 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") pod \"maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:20.900112 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:20.900074 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:21.033386 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:21.033240 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:57:21.035420 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:57:21.035394 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fce541a_8019_415b_9f84_88a7b12a3c00.slice/crio-00a8d4c86bc726b5bcf0f28ba2cea110ea2f67abf39c4246af4ff921c164a25f WatchSource:0}: Error finding container 00a8d4c86bc726b5bcf0f28ba2cea110ea2f67abf39c4246af4ff921c164a25f: Status 404 returned error can't find the container with id 00a8d4c86bc726b5bcf0f28ba2cea110ea2f67abf39c4246af4ff921c164a25f Jul 23 00:57:21.676386 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:21.676352 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" event={"ID":"8fce541a-8019-415b-9f84-88a7b12a3c00","Type":"ContainerStarted","Data":"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d"} Jul 23 00:57:21.676611 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:21.676401 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" event={"ID":"8fce541a-8019-415b-9f84-88a7b12a3c00","Type":"ContainerStarted","Data":"00a8d4c86bc726b5bcf0f28ba2cea110ea2f67abf39c4246af4ff921c164a25f"} Jul 23 00:57:21.676611 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:21.676441 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:21.695185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:21.695145 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" podStartSLOduration=2.244928444 podStartE2EDuration="2.695129732s" totalImagesPullingTime="450.201288ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:57:21.693315199 +0000 UTC m=+2232.999463213" watchObservedRunningTime="2026-07-23 00:57:21.695129732 +0000 UTC m=+2233.001277725" Jul 23 00:57:27.687140 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:27.687106 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:57:48.422573 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.422498 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:48.423069 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.422714 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" podUID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" containerName="maas-api" containerID="cri-o://79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c" gracePeriod=30 Jul 23 00:57:48.664609 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.664588 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:48.819238 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.819205 2575 generic.go:356] "Generic (PLEG): container finished" podID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" containerID="79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c" exitCode=0 Jul 23 00:57:48.819401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.819253 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" Jul 23 00:57:48.819401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.819295 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" event={"ID":"4c4d063b-5d27-406b-af3b-c2fcc90677ee","Type":"ContainerDied","Data":"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c"} Jul 23 00:57:48.819401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.819353 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph" event={"ID":"4c4d063b-5d27-406b-af3b-c2fcc90677ee","Type":"ContainerDied","Data":"771e63222202390f14303ead85ae032e53e599f55da196b5c18f503741d671a0"} Jul 23 00:57:48.819401 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.819381 2575 scope.go:117] "RemoveContainer" containerID="79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c" Jul 23 00:57:48.823049 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.823012 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca\") pod \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " Jul 23 00:57:48.823181 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.823095 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zls56\" (UniqueName: \"kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56\") pod \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " Jul 23 00:57:48.823181 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.823174 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") pod \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\" (UID: \"4c4d063b-5d27-406b-af3b-c2fcc90677ee\") " Jul 23 00:57:48.824800 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.824763 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "4c4d063b-5d27-406b-af3b-c2fcc90677ee" (UID: "4c4d063b-5d27-406b-af3b-c2fcc90677ee"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:57:48.830901 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.830875 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "4c4d063b-5d27-406b-af3b-c2fcc90677ee" (UID: "4c4d063b-5d27-406b-af3b-c2fcc90677ee"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:57:48.831780 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.831743 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56" (OuterVolumeSpecName: "kube-api-access-zls56") pod "4c4d063b-5d27-406b-af3b-c2fcc90677ee" (UID: "4c4d063b-5d27-406b-af3b-c2fcc90677ee"). InnerVolumeSpecName "kube-api-access-zls56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:57:48.845185 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.845164 2575 scope.go:117] "RemoveContainer" containerID="79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c" Jul 23 00:57:48.845480 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:57:48.845460 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c\": container with ID starting with 79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c not found: ID does not exist" containerID="79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c" Jul 23 00:57:48.845543 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.845486 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c"} err="failed to get container status \"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c\": rpc error: code = NotFound desc = could not find container \"79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c\": container with ID starting with 79e5ff153ea41e7f86b0d55ff5a5e6c8d18f1748317d0ba90e5ea424cdd2742c not found: ID does not exist" Jul 23 00:57:48.924252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.924230 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4c4d063b-5d27-406b-af3b-c2fcc90677ee-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:57:48.924252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.924250 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zls56\" (UniqueName: \"kubernetes.io/projected/4c4d063b-5d27-406b-af3b-c2fcc90677ee-kube-api-access-zls56\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:57:48.924381 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:48.924260 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4c4d063b-5d27-406b-af3b-c2fcc90677ee-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:57:49.140342 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:49.140318 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:49.143878 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:49.143856 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-a-23cf83-8975ffcb7-nvvph"] Jul 23 00:57:49.237072 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:57:49.237050 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" path="/var/lib/kubelet/pods/4c4d063b-5d27-406b-af3b-c2fcc90677ee/volumes" Jul 23 00:58:14.977802 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:14.977767 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:58:14.978416 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:14.978032 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" podUID="8fce541a-8019-415b-9f84-88a7b12a3c00" containerName="maas-api" containerID="cri-o://10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d" gracePeriod=30 Jul 23 00:58:15.240786 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.240729 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:58:15.349877 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.349821 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca\") pod \"8fce541a-8019-415b-9f84-88a7b12a3c00\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " Jul 23 00:58:15.350026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.349886 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") pod \"8fce541a-8019-415b-9f84-88a7b12a3c00\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " Jul 23 00:58:15.350026 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.349904 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xntnm\" (UniqueName: \"kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm\") pod \"8fce541a-8019-415b-9f84-88a7b12a3c00\" (UID: \"8fce541a-8019-415b-9f84-88a7b12a3c00\") " Jul 23 00:58:15.350211 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.350185 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8fce541a-8019-415b-9f84-88a7b12a3c00" (UID: "8fce541a-8019-415b-9f84-88a7b12a3c00"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 00:58:15.351936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.351911 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8fce541a-8019-415b-9f84-88a7b12a3c00" (UID: "8fce541a-8019-415b-9f84-88a7b12a3c00"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 00:58:15.351936 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.351908 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm" (OuterVolumeSpecName: "kube-api-access-xntnm") pod "8fce541a-8019-415b-9f84-88a7b12a3c00" (UID: "8fce541a-8019-415b-9f84-88a7b12a3c00"). InnerVolumeSpecName "kube-api-access-xntnm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 00:58:15.451410 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.451383 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8fce541a-8019-415b-9f84-88a7b12a3c00-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:58:15.451410 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.451404 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8fce541a-8019-415b-9f84-88a7b12a3c00-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:58:15.451410 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.451413 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xntnm\" (UniqueName: \"kubernetes.io/projected/8fce541a-8019-415b-9f84-88a7b12a3c00-kube-api-access-xntnm\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 00:58:15.964046 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.964011 2575 generic.go:356] "Generic (PLEG): container finished" podID="8fce541a-8019-415b-9f84-88a7b12a3c00" containerID="10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d" exitCode=0 Jul 23 00:58:15.964218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.964055 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" Jul 23 00:58:15.964218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.964078 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" event={"ID":"8fce541a-8019-415b-9f84-88a7b12a3c00","Type":"ContainerDied","Data":"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d"} Jul 23 00:58:15.964218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.964117 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45" event={"ID":"8fce541a-8019-415b-9f84-88a7b12a3c00","Type":"ContainerDied","Data":"00a8d4c86bc726b5bcf0f28ba2cea110ea2f67abf39c4246af4ff921c164a25f"} Jul 23 00:58:15.964218 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.964139 2575 scope.go:117] "RemoveContainer" containerID="10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d" Jul 23 00:58:15.973461 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.973442 2575 scope.go:117] "RemoveContainer" containerID="10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d" Jul 23 00:58:15.973687 ip-10-0-133-87 kubenswrapper[2575]: E0723 00:58:15.973669 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d\": container with ID starting with 10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d not found: ID does not exist" containerID="10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d" Jul 23 00:58:15.973738 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.973695 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d"} err="failed to get container status \"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d\": rpc error: code = NotFound desc = could not find container \"10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d\": container with ID starting with 10b1d008506a7ba2b72f30cfddc1a5aafebd12f8221c9aa43d294b2afc3ef08d not found: ID does not exist" Jul 23 00:58:15.986458 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.986438 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:58:15.990404 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:15.990382 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-api-b-e6fe2f-5f9cb96985-g8v45"] Jul 23 00:58:17.236691 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:17.236658 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fce541a-8019-415b-9f84-88a7b12a3c00" path="/var/lib/kubelet/pods/8fce541a-8019-415b-9f84-88a7b12a3c00/volumes" Jul 23 00:58:30.508435 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508400 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508782 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fce541a-8019-415b-9f84-88a7b12a3c00" containerName="maas-api" Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508793 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fce541a-8019-415b-9f84-88a7b12a3c00" containerName="maas-api" Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508811 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" containerName="maas-api" Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508817 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" containerName="maas-api" Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508906 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fce541a-8019-415b-9f84-88a7b12a3c00" containerName="maas-api" Jul 23 00:58:30.508919 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.508917 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c4d063b-5d27-406b-af3b-c2fcc90677ee" containerName="maas-api" Jul 23 00:58:30.513153 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.513135 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.515853 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.515814 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-a-b90c57-serving-cert\"" Jul 23 00:58:30.524314 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.524292 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 00:58:30.578082 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.578048 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.578252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.578100 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.578252 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.578155 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4zdhs\" (UniqueName: \"kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.679603 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.679571 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.679763 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.679620 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4zdhs\" (UniqueName: \"kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.679824 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.679802 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.680424 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.680400 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.681962 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.681943 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.688991 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.688971 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4zdhs\" (UniqueName: \"kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs\") pod \"maas-api-e2e-shared-a-b90c57-9fc874445-wn56b\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.825096 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.825076 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:30.958715 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:30.958684 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 00:58:30.960541 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:58:30.960514 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5dc0d018_fb46_4eb0_99f2_000cbf011e66.slice/crio-c66ecb479510f031a5bf3b4858707d36e3f0ec092089c6a34b3b8fdbec9d5978 WatchSource:0}: Error finding container c66ecb479510f031a5bf3b4858707d36e3f0ec092089c6a34b3b8fdbec9d5978: Status 404 returned error can't find the container with id c66ecb479510f031a5bf3b4858707d36e3f0ec092089c6a34b3b8fdbec9d5978 Jul 23 00:58:31.047353 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:31.047325 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" event={"ID":"5dc0d018-fb46-4eb0-99f2-000cbf011e66","Type":"ContainerStarted","Data":"c66ecb479510f031a5bf3b4858707d36e3f0ec092089c6a34b3b8fdbec9d5978"} Jul 23 00:58:32.054540 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:32.054502 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" event={"ID":"5dc0d018-fb46-4eb0-99f2-000cbf011e66","Type":"ContainerStarted","Data":"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5"} Jul 23 00:58:32.055032 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:32.054632 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:32.073246 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:32.073196 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" podStartSLOduration=1.556521712 podStartE2EDuration="2.073183525s" totalImagesPullingTime="516.661813ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:58:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:58:32.072008878 +0000 UTC m=+2303.378156892" watchObservedRunningTime="2026-07-23 00:58:32.073183525 +0000 UTC m=+2303.379331518" Jul 23 00:58:38.065917 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.065891 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 00:58:38.465221 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.465131 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 00:58:38.468993 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.468971 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.471939 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.471919 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-75b665-serving-cert\"" Jul 23 00:58:38.481894 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.481873 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 00:58:38.543007 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.542974 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.543172 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.543114 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pcmlh\" (UniqueName: \"kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.543235 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.543184 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.643789 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.643753 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.643953 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.643817 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pcmlh\" (UniqueName: \"kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.643953 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.643871 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.644421 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.644390 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.646224 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.646206 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.653516 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.653495 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pcmlh\" (UniqueName: \"kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh\") pod \"maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.779975 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.779893 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:38.913226 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:38.913190 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 00:58:38.914524 ip-10-0-133-87 kubenswrapper[2575]: W0723 00:58:38.914500 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ce47a11_1122_4dfd_b559_d6edfa06ed59.slice/crio-6cdb936bc3e54682271ccbdcaddc1adf67d300a45395d10dcac95bba57efc009 WatchSource:0}: Error finding container 6cdb936bc3e54682271ccbdcaddc1adf67d300a45395d10dcac95bba57efc009: Status 404 returned error can't find the container with id 6cdb936bc3e54682271ccbdcaddc1adf67d300a45395d10dcac95bba57efc009 Jul 23 00:58:39.093274 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:39.093239 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" event={"ID":"1ce47a11-1122-4dfd-b559-d6edfa06ed59","Type":"ContainerStarted","Data":"6cdb936bc3e54682271ccbdcaddc1adf67d300a45395d10dcac95bba57efc009"} Jul 23 00:58:40.100290 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:40.100254 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" event={"ID":"1ce47a11-1122-4dfd-b559-d6edfa06ed59","Type":"ContainerStarted","Data":"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a"} Jul 23 00:58:40.100673 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:40.100371 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 00:58:40.118881 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:40.118820 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" podStartSLOduration=1.618266723 podStartE2EDuration="2.118805946s" totalImagesPullingTime="500.539223ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 00:58:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 00:58:40.117746476 +0000 UTC m=+2311.423894490" watchObservedRunningTime="2026-07-23 00:58:40.118805946 +0000 UTC m=+2311.424953963" Jul 23 00:58:46.112220 ip-10-0-133-87 kubenswrapper[2575]: I0723 00:58:46.112188 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 01:00:09.321964 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:00:09.321933 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 01:00:09.330551 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:00:09.330532 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 01:01:38.765800 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.765767 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:01:38.769027 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.769012 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.773265 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.773246 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b90c57\"/\"sub-test-model-b90c57-kserve-self-signed-certs\"" Jul 23 01:01:38.773265 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.773255 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b90c57\"/\"openshift-service-ca.crt\"" Jul 23 01:01:38.773448 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.773287 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b90c57\"/\"kube-root-ca.crt\"" Jul 23 01:01:38.773448 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.773249 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b90c57\"/\"default-dockercfg-5tkvb\"" Jul 23 01:01:38.777704 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.777682 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:01:38.877816 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.877784 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.877978 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.877862 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.877978 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.877902 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.877978 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.877925 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2qh\" (UniqueName: \"kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.878091 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.877981 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.878091 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.878030 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979348 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979320 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979511 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979362 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979511 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979385 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2qh\" (UniqueName: \"kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979511 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979414 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979674 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979516 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979674 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979549 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979800 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979777 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979886 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979810 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.979935 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.979895 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.981563 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.981543 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.981870 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.981849 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:38.987846 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:38.987809 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2qh\" (UniqueName: \"kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh\") pod \"sub-test-model-b90c57-kserve-6d876b48c-tm7sk\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:39.079148 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:39.079123 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:39.212758 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:39.212735 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:01:39.214003 ip-10-0-133-87 kubenswrapper[2575]: W0723 01:01:39.213979 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda2e55cf2_90e7_47fa_af97_cb84b3062cf1.slice/crio-5f16b79678faa4c606d163ad8a805d1269db86072d3536e87b4d0560dd75750e WatchSource:0}: Error finding container 5f16b79678faa4c606d163ad8a805d1269db86072d3536e87b4d0560dd75750e: Status 404 returned error can't find the container with id 5f16b79678faa4c606d163ad8a805d1269db86072d3536e87b4d0560dd75750e Jul 23 01:01:40.062083 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:40.062045 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" event={"ID":"a2e55cf2-90e7-47fa-af97-cb84b3062cf1","Type":"ContainerStarted","Data":"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab"} Jul 23 01:01:40.062461 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:40.062094 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" event={"ID":"a2e55cf2-90e7-47fa-af97-cb84b3062cf1","Type":"ContainerStarted","Data":"5f16b79678faa4c606d163ad8a805d1269db86072d3536e87b4d0560dd75750e"} Jul 23 01:01:40.080682 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:40.080635 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" podStartSLOduration=2.08062224 podStartE2EDuration="2.08062224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-23 01:01:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:01:40.079491364 +0000 UTC m=+2491.385639358" watchObservedRunningTime="2026-07-23 01:01:40.08062224 +0000 UTC m=+2491.386770234" Jul 23 01:01:49.079722 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:49.079685 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:49.080205 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:49.079736 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:49.092400 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:49.092376 2575 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:01:49.123328 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:01:49.123283 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:02:52.663104 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:02:52.663071 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:03:07.682850 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:07.682777 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" podUID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" containerName="main" containerID="cri-o://c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab" gracePeriod=45 Jul 23 01:03:07.923156 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:07.923133 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:03:08.045205 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045130 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045205 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045171 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dq2qh\" (UniqueName: \"kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045205 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045188 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045483 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045211 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045483 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045327 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045483 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045364 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache\") pod \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\" (UID: \"a2e55cf2-90e7-47fa-af97-cb84b3062cf1\") " Jul 23 01:03:08.045483 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045382 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 01:03:08.045713 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045598 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home" (OuterVolumeSpecName: "home") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 01:03:08.045713 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045653 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache" (OuterVolumeSpecName: "model-cache") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 01:03:08.045713 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045709 2575 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-home\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.046443 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.045731 2575 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tmp-dir\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.047299 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.047274 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 01:03:08.047415 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.047316 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh" (OuterVolumeSpecName: "kube-api-access-dq2qh") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "kube-api-access-dq2qh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 01:03:08.047415 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.047379 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm" (OuterVolumeSpecName: "dshm") pod "a2e55cf2-90e7-47fa-af97-cb84b3062cf1" (UID: "a2e55cf2-90e7-47fa-af97-cb84b3062cf1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 23 01:03:08.146552 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.146523 2575 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-model-cache\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.146552 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.146547 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dq2qh\" (UniqueName: \"kubernetes.io/projected/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-kube-api-access-dq2qh\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.146552 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.146558 2575 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-tls-certs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.146757 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.146567 2575 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a2e55cf2-90e7-47fa-af97-cb84b3062cf1-dshm\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:03:08.530279 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.530244 2575 generic.go:356] "Generic (PLEG): container finished" podID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" containerID="c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab" exitCode=0 Jul 23 01:03:08.530442 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.530290 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" Jul 23 01:03:08.530442 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.530328 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" event={"ID":"a2e55cf2-90e7-47fa-af97-cb84b3062cf1","Type":"ContainerDied","Data":"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab"} Jul 23 01:03:08.530442 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.530371 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk" event={"ID":"a2e55cf2-90e7-47fa-af97-cb84b3062cf1","Type":"ContainerDied","Data":"5f16b79678faa4c606d163ad8a805d1269db86072d3536e87b4d0560dd75750e"} Jul 23 01:03:08.530442 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.530389 2575 scope.go:117] "RemoveContainer" containerID="c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab" Jul 23 01:03:08.540030 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.540011 2575 scope.go:117] "RemoveContainer" containerID="c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab" Jul 23 01:03:08.540290 ip-10-0-133-87 kubenswrapper[2575]: E0723 01:03:08.540269 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab\": container with ID starting with c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab not found: ID does not exist" containerID="c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab" Jul 23 01:03:08.540362 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.540304 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab"} err="failed to get container status \"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab\": rpc error: code = NotFound desc = could not find container \"c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab\": container with ID starting with c43069103cb72e1066e3e7f80d4694fa358e82996474370808ad344b0c0271ab not found: ID does not exist" Jul 23 01:03:08.552122 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.552100 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:03:08.555529 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:08.555508 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b90c57/sub-test-model-b90c57-kserve-6d876b48c-tm7sk"] Jul 23 01:03:09.242280 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:03:09.242243 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" path="/var/lib/kubelet/pods/a2e55cf2-90e7-47fa-af97-cb84b3062cf1/volumes" Jul 23 01:05:09.356148 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:09.356034 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 01:05:09.372784 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:09.372761 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-fhw25_9e16c15a-e9ce-4d4a-a41a-bc6c1cfb759c/ovn-acl-logging/0.log" Jul 23 01:05:27.496489 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.496455 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 01:05:27.497042 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.496656 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" podUID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" containerName="maas-api" containerID="cri-o://0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5" gracePeriod=30 Jul 23 01:05:27.747647 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.747592 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 01:05:27.834746 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.834715 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4zdhs\" (UniqueName: \"kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs\") pod \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " Jul 23 01:05:27.834903 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.834760 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls\") pod \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " Jul 23 01:05:27.834903 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.834821 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca\") pod \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\" (UID: \"5dc0d018-fb46-4eb0-99f2-000cbf011e66\") " Jul 23 01:05:27.835210 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.835187 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5dc0d018-fb46-4eb0-99f2-000cbf011e66" (UID: "5dc0d018-fb46-4eb0-99f2-000cbf011e66"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 01:05:27.836738 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.836713 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5dc0d018-fb46-4eb0-99f2-000cbf011e66" (UID: "5dc0d018-fb46-4eb0-99f2-000cbf011e66"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 01:05:27.836738 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.836732 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs" (OuterVolumeSpecName: "kube-api-access-4zdhs") pod "5dc0d018-fb46-4eb0-99f2-000cbf011e66" (UID: "5dc0d018-fb46-4eb0-99f2-000cbf011e66"). InnerVolumeSpecName "kube-api-access-4zdhs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 01:05:27.936010 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.935982 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4zdhs\" (UniqueName: \"kubernetes.io/projected/5dc0d018-fb46-4eb0-99f2-000cbf011e66-kube-api-access-4zdhs\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:27.936010 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.936005 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5dc0d018-fb46-4eb0-99f2-000cbf011e66-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:27.936152 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:27.936014 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5dc0d018-fb46-4eb0-99f2-000cbf011e66-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:28.265175 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.265144 2575 generic.go:356] "Generic (PLEG): container finished" podID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" containerID="0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5" exitCode=0 Jul 23 01:05:28.265319 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.265181 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" Jul 23 01:05:28.265319 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.265191 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" event={"ID":"5dc0d018-fb46-4eb0-99f2-000cbf011e66","Type":"ContainerDied","Data":"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5"} Jul 23 01:05:28.265319 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.265231 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b" event={"ID":"5dc0d018-fb46-4eb0-99f2-000cbf011e66","Type":"ContainerDied","Data":"c66ecb479510f031a5bf3b4858707d36e3f0ec092089c6a34b3b8fdbec9d5978"} Jul 23 01:05:28.265319 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.265252 2575 scope.go:117] "RemoveContainer" containerID="0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5" Jul 23 01:05:28.274462 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.274444 2575 scope.go:117] "RemoveContainer" containerID="0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5" Jul 23 01:05:28.274725 ip-10-0-133-87 kubenswrapper[2575]: E0723 01:05:28.274694 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5\": container with ID starting with 0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5 not found: ID does not exist" containerID="0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5" Jul 23 01:05:28.274775 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.274734 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5"} err="failed to get container status \"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5\": rpc error: code = NotFound desc = could not find container \"0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5\": container with ID starting with 0f725b8d276269965ca87fba0c4d38d642ff8da0b5eba37b9a528fe14aab22b5 not found: ID does not exist" Jul 23 01:05:28.286775 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.286754 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 01:05:28.290286 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:28.290262 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-a-b90c57-9fc874445-wn56b"] Jul 23 01:05:29.237717 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:29.237682 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" path="/var/lib/kubelet/pods/5dc0d018-fb46-4eb0-99f2-000cbf011e66/volumes" Jul 23 01:05:49.956701 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:49.956666 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 01:05:49.957176 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:49.956943 2575 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" podUID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" containerName="maas-api" containerID="cri-o://da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a" gracePeriod=30 Jul 23 01:05:50.199057 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.199029 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 01:05:50.226328 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.226271 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pcmlh\" (UniqueName: \"kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh\") pod \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " Jul 23 01:05:50.226436 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.226382 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls\") pod \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " Jul 23 01:05:50.226436 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.226410 2575 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca\") pod \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\" (UID: \"1ce47a11-1122-4dfd-b559-d6edfa06ed59\") " Jul 23 01:05:50.226865 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.226840 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "1ce47a11-1122-4dfd-b559-d6edfa06ed59" (UID: "1ce47a11-1122-4dfd-b559-d6edfa06ed59"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 23 01:05:50.228267 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.228239 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh" (OuterVolumeSpecName: "kube-api-access-pcmlh") pod "1ce47a11-1122-4dfd-b559-d6edfa06ed59" (UID: "1ce47a11-1122-4dfd-b559-d6edfa06ed59"). InnerVolumeSpecName "kube-api-access-pcmlh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 23 01:05:50.228346 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.228299 2575 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "1ce47a11-1122-4dfd-b559-d6edfa06ed59" (UID: "1ce47a11-1122-4dfd-b559-d6edfa06ed59"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 23 01:05:50.327671 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.327646 2575 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/1ce47a11-1122-4dfd-b559-d6edfa06ed59-maas-api-tls\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:50.327671 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.327669 2575 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/1ce47a11-1122-4dfd-b559-d6edfa06ed59-openshift-service-ca\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:50.327801 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.327679 2575 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pcmlh\" (UniqueName: \"kubernetes.io/projected/1ce47a11-1122-4dfd-b559-d6edfa06ed59-kube-api-access-pcmlh\") on node \"ip-10-0-133-87.ec2.internal\" DevicePath \"\"" Jul 23 01:05:50.382579 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.382550 2575 generic.go:356] "Generic (PLEG): container finished" podID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" containerID="da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a" exitCode=0 Jul 23 01:05:50.382667 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.382594 2575 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" Jul 23 01:05:50.382667 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.382597 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" event={"ID":"1ce47a11-1122-4dfd-b559-d6edfa06ed59","Type":"ContainerDied","Data":"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a"} Jul 23 01:05:50.382667 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.382649 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s" event={"ID":"1ce47a11-1122-4dfd-b559-d6edfa06ed59","Type":"ContainerDied","Data":"6cdb936bc3e54682271ccbdcaddc1adf67d300a45395d10dcac95bba57efc009"} Jul 23 01:05:50.382771 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.382672 2575 scope.go:117] "RemoveContainer" containerID="da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a" Jul 23 01:05:50.392661 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.392639 2575 scope.go:117] "RemoveContainer" containerID="da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a" Jul 23 01:05:50.393031 ip-10-0-133-87 kubenswrapper[2575]: E0723 01:05:50.393012 2575 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a\": container with ID starting with da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a not found: ID does not exist" containerID="da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a" Jul 23 01:05:50.393100 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.393039 2575 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a"} err="failed to get container status \"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a\": rpc error: code = NotFound desc = could not find container \"da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a\": container with ID starting with da55c87fae87337e42249cbb5edda6ff27a0ce51651c43eb9ca30fa5b68fa82a not found: ID does not exist" Jul 23 01:05:50.405362 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.405341 2575 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 01:05:50.408620 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:50.408599 2575 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-75b665-86989cfcc9-ktd6s"] Jul 23 01:05:51.236767 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:05:51.236730 2575 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" path="/var/lib/kubelet/pods/1ce47a11-1122-4dfd-b559-d6edfa06ed59/volumes" Jul 23 01:06:07.641992 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:07.641962 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6487687876-pk67l_38933b96-ddd8-4778-ab7d-a8e4486ffcb4/manager/0.log" Jul 23 01:06:08.147380 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:08.147348 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8844c5db8-d6tdk_8394b76f-9271-4d86-b2dc-ec9d08b6ab26/copy-manifests/0.log" Jul 23 01:06:08.161690 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:08.161665 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8844c5db8-d6tdk_8394b76f-9271-4d86-b2dc-ec9d08b6ab26/manager/0.log" Jul 23 01:06:08.649354 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:08.649324 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-8676f89cb7-bgjmv_eafbce22-d6a2-4c10-b98b-7e06c40a39ef/maas-api/0.log" Jul 23 01:06:09.114255 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:09.114222 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-t2vhq_1cb17166-7167-4127-8bcd-7ff01a00a3bb/postgres/0.log" Jul 23 01:06:09.945593 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:09.945563 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/pull/0.log" Jul 23 01:06:09.951787 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:09.951761 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/extract/0.log" Jul 23 01:06:09.957669 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:09.957649 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/util/0.log" Jul 23 01:06:10.074236 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.074205 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/util/0.log" Jul 23 01:06:10.080629 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.080591 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/pull/0.log" Jul 23 01:06:10.086837 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.086806 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/extract/0.log" Jul 23 01:06:10.203541 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.203473 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/util/0.log" Jul 23 01:06:10.210114 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.210081 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/pull/0.log" Jul 23 01:06:10.216641 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.216624 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/extract/0.log" Jul 23 01:06:10.338398 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.338369 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/util/0.log" Jul 23 01:06:10.344718 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.344693 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/pull/0.log" Jul 23 01:06:10.351824 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:10.351808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/extract/0.log" Jul 23 01:06:11.898305 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:11.898269 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qcg8k_6a58d0bb-9fa6-491c-a88a-8f980619d0b1/discovery/0.log" Jul 23 01:06:12.407123 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:12.407093 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wstk6_80798469-9bb3-409e-8b9e-9d4fe1b2c1e3/payload-pre-processing/0.log" Jul 23 01:06:12.536050 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:12.536020 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-67fgb_f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89/payload-processing/0.log" Jul 23 01:06:13.018257 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:13.018224 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-267zp_eb645107-c098-444c-9c68-6b796dc46f1d/main/0.log" Jul 23 01:06:13.258764 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:13.258737 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-hwkq4_70ac46ac-6046-46dc-9670-dc7b1c534924/main/0.log" Jul 23 01:06:13.495539 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:13.495514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5qg4c_dab34665-c096-4644-86de-7183825ce4fb/main/0.log" Jul 23 01:06:22.447256 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:22.447223 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-htfz9_22e1b738-7df0-4e85-99a9-335b1abd8c66/global-pull-secret-syncer/0.log" Jul 23 01:06:22.654415 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:22.654386 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ztl74_9b5bab9e-600d-40e6-a6f2-57cb9fcf0bef/konnectivity-agent/0.log" Jul 23 01:06:22.674810 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:22.674788 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-87.ec2.internal_a9ca0bc95fbca063e66eb3b93e200e1c/haproxy/0.log" Jul 23 01:06:27.142191 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.142165 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/extract/0.log" Jul 23 01:06:27.170141 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.170115 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/util/0.log" Jul 23 01:06:27.193467 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.193440 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c5zqf_d18608ad-add2-4401-aceb-e7cecde3ea08/pull/0.log" Jul 23 01:06:27.245635 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.245611 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/extract/0.log" Jul 23 01:06:27.268633 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.268608 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/util/0.log" Jul 23 01:06:27.292624 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.292595 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0mpmxd_35c022c0-1485-4a3e-ae0b-964f4b17a244/pull/0.log" Jul 23 01:06:27.324120 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.324099 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/extract/0.log" Jul 23 01:06:27.346819 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.346802 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/util/0.log" Jul 23 01:06:27.369201 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.369183 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed738pqhg_0e53bc96-13d5-4716-989e-c2998b8042fd/pull/0.log" Jul 23 01:06:27.397766 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.397696 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/extract/0.log" Jul 23 01:06:27.426032 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.426011 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/util/0.log" Jul 23 01:06:27.448975 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:27.448937 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef16wbx2_51372e8b-ba0d-41f1-9535-706617fe2a3e/pull/0.log" Jul 23 01:06:29.728972 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.728939 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/alertmanager/0.log" Jul 23 01:06:29.759388 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.759367 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/config-reloader/0.log" Jul 23 01:06:29.795191 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.795172 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/kube-rbac-proxy-web/0.log" Jul 23 01:06:29.825026 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.824995 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/kube-rbac-proxy/0.log" Jul 23 01:06:29.857518 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.857497 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/kube-rbac-proxy-metric/0.log" Jul 23 01:06:29.886655 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.886633 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/prom-label-proxy/0.log" Jul 23 01:06:29.917434 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:29.917416 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_7bff247c-874d-49f0-a1a6-5333b681f321/init-config-reloader/0.log" Jul 23 01:06:30.110451 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.110412 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-tgbb2_e81ed6d1-5380-41de-be29-e1a7988c27a5/monitoring-plugin/0.log" Jul 23 01:06:30.161675 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.161658 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fv2b4_ef8a023b-bd72-443a-9653-cba071e65ec4/node-exporter/0.log" Jul 23 01:06:30.190509 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.190493 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fv2b4_ef8a023b-bd72-443a-9653-cba071e65ec4/kube-rbac-proxy/0.log" Jul 23 01:06:30.212380 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.212364 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-fv2b4_ef8a023b-bd72-443a-9653-cba071e65ec4/init-textfile/0.log" Jul 23 01:06:30.398598 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.398522 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-67qkf_2ef2889d-494b-4c1f-a417-c95225c9e358/kube-rbac-proxy-main/0.log" Jul 23 01:06:30.420446 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.420427 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-67qkf_2ef2889d-494b-4c1f-a417-c95225c9e358/kube-rbac-proxy-self/0.log" Jul 23 01:06:30.444122 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.444099 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-67qkf_2ef2889d-494b-4c1f-a417-c95225c9e358/openshift-state-metrics/0.log" Jul 23 01:06:30.740125 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.740020 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5fcfd56f-5dwq9_83d083a1-f8e1-4851-8271-d9f6ea276e19/telemeter-client/0.log" Jul 23 01:06:30.762207 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.762178 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5fcfd56f-5dwq9_83d083a1-f8e1-4851-8271-d9f6ea276e19/reload/0.log" Jul 23 01:06:30.784089 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.784069 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-d5fcfd56f-5dwq9_83d083a1-f8e1-4851-8271-d9f6ea276e19/kube-rbac-proxy/0.log" Jul 23 01:06:30.824059 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.824043 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/thanos-query/0.log" Jul 23 01:06:30.846406 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.846390 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/kube-rbac-proxy-web/0.log" Jul 23 01:06:30.874489 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.874459 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/kube-rbac-proxy/0.log" Jul 23 01:06:30.899478 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.899457 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/prom-label-proxy/0.log" Jul 23 01:06:30.921757 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.921720 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/kube-rbac-proxy-rules/0.log" Jul 23 01:06:30.945806 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:30.945790 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7565bff569-k78sx_87252b26-97dc-4e17-8f1b-6ae2d907b90c/kube-rbac-proxy-metrics/0.log" Jul 23 01:06:32.448746 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.448712 2575 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv"] Jul 23 01:06:32.449223 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449199 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" containerName="main" Jul 23 01:06:32.449223 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449223 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" containerName="main" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449249 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449258 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449284 2575 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449292 2575 state_mem.go:107] "Deleted CPUSet assignment" podUID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449382 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="5dc0d018-fb46-4eb0-99f2-000cbf011e66" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449398 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="1ce47a11-1122-4dfd-b559-d6edfa06ed59" containerName="maas-api" Jul 23 01:06:32.449422 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.449408 2575 memory_manager.go:368] "RemoveStaleState removing state" podUID="a2e55cf2-90e7-47fa-af97-cb84b3062cf1" containerName="main" Jul 23 01:06:32.452488 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.452473 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.454995 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.454973 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rz22l\"/\"kube-root-ca.crt\"" Jul 23 01:06:32.455121 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.455078 2575 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rz22l\"/\"default-dockercfg-htbdw\"" Jul 23 01:06:32.455189 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.455166 2575 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rz22l\"/\"openshift-service-ca.crt\"" Jul 23 01:06:32.462076 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.462047 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv"] Jul 23 01:06:32.589767 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.589741 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-sys\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.589952 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.589775 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-proc\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.589952 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.589814 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-podres\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.589952 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.589919 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-lib-modules\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.590077 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.589958 2575 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4m6z9\" (UniqueName: \"kubernetes.io/projected/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-kube-api-access-4m6z9\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691009 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.690980 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-lib-modules\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691138 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691015 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4m6z9\" (UniqueName: \"kubernetes.io/projected/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-kube-api-access-4m6z9\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691138 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691054 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-sys\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691138 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691084 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-proc\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691294 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691148 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-proc\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691294 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691157 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-lib-modules\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691294 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691162 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-sys\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691294 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691149 2575 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-podres\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.691294 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.691235 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-podres\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.698541 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.698515 2575 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4m6z9\" (UniqueName: \"kubernetes.io/projected/e9fc26bd-48ec-4ddf-9ef5-48d2827edaca-kube-api-access-4m6z9\") pod \"perf-node-gather-daemonset-wcgvv\" (UID: \"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca\") " pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.763499 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.763446 2575 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:32.891158 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:32.891127 2575 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv"] Jul 23 01:06:32.896039 ip-10-0-133-87 kubenswrapper[2575]: W0723 01:06:32.893293 2575 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode9fc26bd_48ec_4ddf_9ef5_48d2827edaca.slice/crio-60fa8e886fc25f64460aea10231c14aaf11762d815585682cf5987abe2386641 WatchSource:0}: Error finding container 60fa8e886fc25f64460aea10231c14aaf11762d815585682cf5987abe2386641: Status 404 returned error can't find the container with id 60fa8e886fc25f64460aea10231c14aaf11762d815585682cf5987abe2386641 Jul 23 01:06:33.452336 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:33.452311 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5f448b6977-jhj8l_8581c2c8-5b15-41dd-a2ab-3def63863816/console/0.log" Jul 23 01:06:33.610412 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:33.610381 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" event={"ID":"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca","Type":"ContainerStarted","Data":"52373b1a95a47a13a265f3a4d0a7a0f32c27481bab7162be4fc43c253d99e475"} Jul 23 01:06:33.610578 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:33.610440 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:33.610578 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:33.610456 2575 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" event={"ID":"e9fc26bd-48ec-4ddf-9ef5-48d2827edaca","Type":"ContainerStarted","Data":"60fa8e886fc25f64460aea10231c14aaf11762d815585682cf5987abe2386641"} Jul 23 01:06:33.626901 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:33.626856 2575 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" podStartSLOduration=1.626825042 podStartE2EDuration="1.626825042s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-23 01:06:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-23 01:06:33.625909161 +0000 UTC m=+2784.932057155" watchObservedRunningTime="2026-07-23 01:06:33.626825042 +0000 UTC m=+2784.932973035" Jul 23 01:06:35.154750 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:35.154715 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g55qm_36de04af-c095-46e4-ae46-4b049fda43fc/dns/0.log" Jul 23 01:06:35.178097 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:35.178070 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-g55qm_36de04af-c095-46e4-ae46-4b049fda43fc/kube-rbac-proxy/0.log" Jul 23 01:06:35.360188 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:35.360163 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-ldjk4_ac51900d-e12b-494e-a4e9-5e30ebff971d/dns-node-resolver/0.log" Jul 23 01:06:35.877430 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:35.877403 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5bd67c9c9f-44p27_13b3c005-9f4b-49b2-b193-ac8df029ac6e/registry/0.log" Jul 23 01:06:35.898437 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:35.898409 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-74sfp_e44b9dfb-a0a3-4d0d-be49-2a7fb85e8ba8/node-ca/0.log" Jul 23 01:06:37.066234 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:37.066205 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qcg8k_6a58d0bb-9fa6-491c-a88a-8f980619d0b1/discovery/0.log" Jul 23 01:06:37.218763 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:37.218737 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wstk6_80798469-9bb3-409e-8b9e-9d4fe1b2c1e3/payload-pre-processing/0.log" Jul 23 01:06:37.249086 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:37.249064 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-67fgb_f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89/payload-processing/0.log" Jul 23 01:06:37.931331 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:37.931305 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gmhmg_5cfcd5ef-d275-4d6a-b7ca-1456b86dd751/serve-healthcheck-canary/0.log" Jul 23 01:06:38.475126 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:38.475097 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6q89s_a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd/kube-rbac-proxy/0.log" Jul 23 01:06:38.498543 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:38.498513 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6q89s_a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd/exporter/0.log" Jul 23 01:06:38.520839 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:38.520808 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-6q89s_a3a0ceb9-8c14-4d1e-bf13-4319d7b6f4bd/extractor/0.log" Jul 23 01:06:39.625233 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:39.625206 2575 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-rz22l/perf-node-gather-daemonset-wcgvv" Jul 23 01:06:41.125781 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:41.125745 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6487687876-pk67l_38933b96-ddd8-4778-ab7d-a8e4486ffcb4/manager/0.log" Jul 23 01:06:41.384868 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:41.384770 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8844c5db8-d6tdk_8394b76f-9271-4d86-b2dc-ec9d08b6ab26/manager/0.log" Jul 23 01:06:41.404532 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:41.404514 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8844c5db8-d6tdk_8394b76f-9271-4d86-b2dc-ec9d08b6ab26/copy-manifests/0.log" Jul 23 01:06:42.942154 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:42.942081 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-6958b76574-rwk2w_e42549f7-001a-449a-9d05-630286b3ba10/manager/0.log" Jul 23 01:06:43.270713 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:43.270637 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qcg8k_6a58d0bb-9fa6-491c-a88a-8f980619d0b1/discovery/0.log" Jul 23 01:06:43.385525 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:43.385496 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wstk6_80798469-9bb3-409e-8b9e-9d4fe1b2c1e3/payload-pre-processing/0.log" Jul 23 01:06:43.409451 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:43.409428 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-67fgb_f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89/payload-processing/0.log" Jul 23 01:06:44.732282 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:44.732252 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-qcg8k_6a58d0bb-9fa6-491c-a88a-8f980619d0b1/discovery/0.log" Jul 23 01:06:44.843794 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:44.843751 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-wstk6_80798469-9bb3-409e-8b9e-9d4fe1b2c1e3/payload-pre-processing/0.log" Jul 23 01:06:44.865114 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:44.865088 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5bf47cd988-67fgb_f3633e4b-62b9-4b7f-9cb1-3da6b2a74c89/payload-processing/0.log" Jul 23 01:06:49.359962 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.359929 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/kube-multus-additional-cni-plugins/0.log" Jul 23 01:06:49.380910 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.380881 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/egress-router-binary-copy/0.log" Jul 23 01:06:49.402277 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.402258 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/cni-plugins/0.log" Jul 23 01:06:49.423746 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.423728 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/bond-cni-plugin/0.log" Jul 23 01:06:49.447369 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.447351 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/routeoverride-cni/0.log" Jul 23 01:06:49.471542 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.471524 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/whereabouts-cni-bincopy/0.log" Jul 23 01:06:49.494335 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.494317 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-25s8z_912ac498-7d64-46a8-bc09-5cb60db5ea55/whereabouts-cni/0.log" Jul 23 01:06:49.866325 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:49.866294 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-q2rrf_9ba635bf-a492-4592-9d61-1666f54a3d7f/kube-multus/0.log" Jul 23 01:06:50.019600 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:50.019570 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-krcjb_562817ed-d6e5-427e-9acf-d9e3804bd86c/network-metrics-daemon/0.log" Jul 23 01:06:50.039961 ip-10-0-133-87 kubenswrapper[2575]: I0723 01:06:50.039938 2575 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-krcjb_562817ed-d6e5-427e-9acf-d9e3804bd86c/kube-rbac-proxy/0.log"