Jul 12 15:18:11.824641 ip-10-0-136-30 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 12 15:18:11.824653 ip-10-0-136-30 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 12 15:18:11.824660 ip-10-0-136-30 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 12 15:18:11.824881 ip-10-0-136-30 systemd[1]: Failed to start Kubernetes Kubelet. Jul 12 15:18:22.029153 ip-10-0-136-30 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 12 15:18:22.029177 ip-10-0-136-30 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot dce85a0feda947c6835d7fcd3efd9c7c -- Jul 12 15:20:22.172364 ip-10-0-136-30 systemd[1]: Starting Kubernetes Kubelet... Jul 12 15:20:22.578129 ip-10-0-136-30 kubenswrapper[2567]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:22.578129 ip-10-0-136-30 kubenswrapper[2567]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 12 15:20:22.578129 ip-10-0-136-30 kubenswrapper[2567]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:22.578129 ip-10-0-136-30 kubenswrapper[2567]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 12 15:20:22.578129 ip-10-0-136-30 kubenswrapper[2567]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 12 15:20:22.578880 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.578193 2567 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587643 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587666 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587669 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587672 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587676 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587679 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587682 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.587674 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587685 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587688 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587691 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587695 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587700 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587704 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587709 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587712 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587715 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587718 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587721 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587723 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587727 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587730 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587733 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587735 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587738 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587740 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587743 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.587955 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587745 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587749 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587752 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587754 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587757 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587759 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587761 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587764 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587767 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587769 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587772 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587775 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587777 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587780 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587783 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587785 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587788 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587791 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587793 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.588434 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587796 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587798 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587801 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587803 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587805 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587809 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587811 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587813 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587816 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587818 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587821 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587823 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587825 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587828 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587830 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587833 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587835 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587837 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587840 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587842 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.588922 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587845 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587848 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587850 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587853 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587856 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587858 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587861 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587863 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587866 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587868 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587871 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587874 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587877 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587879 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587882 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587884 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587887 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587889 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587893 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587895 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.589427 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587899 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587903 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587905 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587909 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587911 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587914 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587917 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587919 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587922 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587925 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587927 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587930 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587933 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.587936 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589420 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589429 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589434 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589437 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589440 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589444 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.589903 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589447 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589449 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589452 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589456 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589460 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589463 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589467 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589471 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589474 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589476 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589479 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589483 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589486 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589489 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589491 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589493 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589496 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589499 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589501 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.590417 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589504 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589506 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589510 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589512 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589515 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589518 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589522 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589525 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589528 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589531 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589534 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589537 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589540 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589543 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589545 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589548 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589550 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589552 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589555 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.590912 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589557 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589560 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589562 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589565 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589567 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589570 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589573 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589576 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589578 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589581 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589584 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589587 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589589 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589592 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589594 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589596 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589599 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589602 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589605 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589608 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.591401 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589611 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589613 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589616 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589618 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589621 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589624 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589626 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589629 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589631 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589633 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589636 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589639 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589641 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589644 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589646 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589649 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589651 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589654 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589656 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589659 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.591923 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589662 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589664 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589667 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589669 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589672 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589674 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589676 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589679 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589681 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589684 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589687 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589689 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589692 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589694 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.589697 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590100 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590106 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590109 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590112 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.592436 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590115 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590117 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590120 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590122 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590125 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590127 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590130 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590133 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590135 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590138 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590140 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590143 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590146 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590148 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590151 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590153 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590155 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590158 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590161 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590166 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.592895 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590168 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590171 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590173 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590176 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590179 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590181 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590184 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590188 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590191 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590194 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590197 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590199 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590202 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590204 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590206 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590210 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590214 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590217 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590220 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.593398 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590223 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590225 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590228 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590230 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590233 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590236 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590240 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590243 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590247 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590250 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590253 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590255 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590259 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590262 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590264 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590267 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590269 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590271 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590274 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.593859 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590276 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590279 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590281 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590284 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590286 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590289 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590291 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590294 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590296 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590299 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590302 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590305 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590308 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590310 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590313 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590315 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590317 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590320 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590322 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590324 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.594338 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590328 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590331 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590333 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590336 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590339 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590342 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590345 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590347 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590349 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590352 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590354 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590357 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590359 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590361 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590364 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590366 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.590368 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590447 2567 flags.go:64] FLAG: --address="0.0.0.0" Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590455 2567 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590462 2567 flags.go:64] FLAG: --anonymous-auth="true" Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590467 2567 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590471 2567 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 12 15:20:22.594815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590474 2567 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590478 2567 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590482 2567 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590486 2567 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590489 2567 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590492 2567 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590496 2567 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590499 2567 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590502 2567 flags.go:64] FLAG: --cgroup-root="" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590504 2567 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590507 2567 flags.go:64] FLAG: --client-ca-file="" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590511 2567 flags.go:64] FLAG: --cloud-provider="external" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590514 2567 flags.go:64] FLAG: --cluster-dns="[]" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590518 2567 flags.go:64] FLAG: --cluster-domain="" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590521 2567 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590526 2567 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590529 2567 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590532 2567 flags.go:64] FLAG: --container-log-max-files="5" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590536 2567 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590539 2567 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590542 2567 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590545 2567 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590548 2567 flags.go:64] FLAG: --contention-profiling="false" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590551 2567 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 12 15:20:22.595400 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590554 2567 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590557 2567 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590560 2567 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590564 2567 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590567 2567 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590569 2567 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590572 2567 flags.go:64] FLAG: --enable-load-reader="false" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590575 2567 flags.go:64] FLAG: --enable-server="true" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590578 2567 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590583 2567 flags.go:64] FLAG: --event-burst="100" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590586 2567 flags.go:64] FLAG: --event-qps="50" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590589 2567 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590592 2567 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590595 2567 flags.go:64] FLAG: --eviction-hard="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590599 2567 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590602 2567 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590605 2567 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590608 2567 flags.go:64] FLAG: --eviction-soft="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590611 2567 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590614 2567 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590618 2567 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590620 2567 flags.go:64] FLAG: --experimental-mounter-path="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590623 2567 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590628 2567 flags.go:64] FLAG: --fail-swap-on="true" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590631 2567 flags.go:64] FLAG: --feature-gates="" Jul 12 15:20:22.595957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590635 2567 flags.go:64] FLAG: --file-check-frequency="20s" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590638 2567 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590640 2567 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590644 2567 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590646 2567 flags.go:64] FLAG: --healthz-port="10248" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590649 2567 flags.go:64] FLAG: --help="false" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590652 2567 flags.go:64] FLAG: --hostname-override="ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590655 2567 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590659 2567 flags.go:64] FLAG: --http-check-frequency="20s" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590662 2567 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590665 2567 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590668 2567 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590671 2567 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590673 2567 flags.go:64] FLAG: --image-service-endpoint="" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590677 2567 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590680 2567 flags.go:64] FLAG: --kube-api-burst="100" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590683 2567 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590686 2567 flags.go:64] FLAG: --kube-api-qps="50" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590688 2567 flags.go:64] FLAG: --kube-reserved="" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590691 2567 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590694 2567 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590697 2567 flags.go:64] FLAG: --kubelet-cgroups="" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590700 2567 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590703 2567 flags.go:64] FLAG: --lock-file="" Jul 12 15:20:22.596722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590706 2567 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590708 2567 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590711 2567 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590717 2567 flags.go:64] FLAG: --log-json-split-stream="false" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590721 2567 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590723 2567 flags.go:64] FLAG: --log-text-split-stream="false" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590728 2567 flags.go:64] FLAG: --logging-format="text" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590731 2567 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590734 2567 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590737 2567 flags.go:64] FLAG: --manifest-url="" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590740 2567 flags.go:64] FLAG: --manifest-url-header="" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590744 2567 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590747 2567 flags.go:64] FLAG: --max-open-files="1000000" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590751 2567 flags.go:64] FLAG: --max-pods="110" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590754 2567 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590757 2567 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590760 2567 flags.go:64] FLAG: --memory-manager-policy="None" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590763 2567 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590765 2567 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590768 2567 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590771 2567 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590780 2567 flags.go:64] FLAG: --node-status-max-images="50" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590783 2567 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590786 2567 flags.go:64] FLAG: --oom-score-adj="-999" Jul 12 15:20:22.597349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590789 2567 flags.go:64] FLAG: --pod-cidr="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590792 2567 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590798 2567 flags.go:64] FLAG: --pod-manifest-path="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590801 2567 flags.go:64] FLAG: --pod-max-pids="-1" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590804 2567 flags.go:64] FLAG: --pods-per-core="0" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590808 2567 flags.go:64] FLAG: --port="10250" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590811 2567 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590814 2567 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0df778e257bb6fee0" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590817 2567 flags.go:64] FLAG: --qos-reserved="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590820 2567 flags.go:64] FLAG: --read-only-port="10255" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590822 2567 flags.go:64] FLAG: --register-node="true" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590825 2567 flags.go:64] FLAG: --register-with-taints="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590828 2567 flags.go:64] FLAG: --registry-burst="10" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590834 2567 flags.go:64] FLAG: --registry-qps="5" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590837 2567 flags.go:64] FLAG: --reserved-cpus="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590842 2567 flags.go:64] FLAG: --reserved-memory="" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590845 2567 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590848 2567 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590851 2567 flags.go:64] FLAG: --rotate-certificates="false" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590854 2567 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590857 2567 flags.go:64] FLAG: --runonce="false" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590860 2567 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590863 2567 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590865 2567 flags.go:64] FLAG: --seccomp-default="false" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590868 2567 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 12 15:20:22.597946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590871 2567 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590874 2567 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590877 2567 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590880 2567 flags.go:64] FLAG: --storage-driver-password="root" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590883 2567 flags.go:64] FLAG: --storage-driver-secure="false" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590886 2567 flags.go:64] FLAG: --storage-driver-table="stats" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590889 2567 flags.go:64] FLAG: --storage-driver-user="root" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590892 2567 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590894 2567 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590899 2567 flags.go:64] FLAG: --system-cgroups="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590902 2567 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590907 2567 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590909 2567 flags.go:64] FLAG: --tls-cert-file="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590912 2567 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590916 2567 flags.go:64] FLAG: --tls-min-version="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590919 2567 flags.go:64] FLAG: --tls-private-key-file="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590922 2567 flags.go:64] FLAG: --topology-manager-policy="none" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590925 2567 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590928 2567 flags.go:64] FLAG: --topology-manager-scope="container" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590931 2567 flags.go:64] FLAG: --v="2" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590935 2567 flags.go:64] FLAG: --version="false" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590940 2567 flags.go:64] FLAG: --vmodule="" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590944 2567 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.590948 2567 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591084 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.598558 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591089 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591091 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591094 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591096 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591099 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591101 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591104 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591106 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591109 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591113 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591116 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591119 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591122 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591125 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591127 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591130 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591134 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591137 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591139 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591141 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.599191 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591144 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591146 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591149 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591151 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591153 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591156 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591158 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591161 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591164 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591168 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591171 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591174 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591176 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591179 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591181 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591184 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591186 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591189 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591191 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591194 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.599679 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591197 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591199 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591202 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591204 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591207 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591209 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591212 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591214 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591218 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591221 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591223 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591226 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591228 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591231 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591233 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591235 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591238 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591241 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591243 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591245 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.600176 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591248 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591250 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591253 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591255 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591258 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591260 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591262 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591265 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591267 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591270 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591272 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591274 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591277 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591279 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591282 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591284 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591287 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591289 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591293 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591296 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.600695 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591301 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591304 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591306 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591310 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591312 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591315 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591318 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591320 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591323 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591326 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591328 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591331 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591333 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591336 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591338 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591341 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591344 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.601254 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.591346 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.591908 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.598853 2567 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.598869 2567 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.599860 2567 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.599868 2567 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599926 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599931 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599934 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599937 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599940 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599943 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599946 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599949 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.601662 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599951 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599955 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599957 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599960 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599962 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599965 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599967 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599970 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599973 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599975 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599978 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599981 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599984 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599986 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599989 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599991 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599995 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.599998 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600000 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600002 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.602007 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600005 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600007 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600010 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600013 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600015 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600017 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600020 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600024 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600027 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600030 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600033 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600036 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600039 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600042 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600057 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600060 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600063 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600065 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600068 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.602502 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600070 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600073 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600075 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600077 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600080 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600082 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600086 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600088 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600091 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600093 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600097 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600099 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600102 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600104 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600107 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600109 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600112 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600114 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600116 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600119 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.602957 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600121 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600124 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600126 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600129 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600131 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600134 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600137 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600141 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600145 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600148 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600150 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600153 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600156 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600159 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600161 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600164 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600167 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600169 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600172 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600174 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.603468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600177 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600180 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600183 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600186 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600189 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600193 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600197 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600200 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600202 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600205 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600207 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600210 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.600215 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600329 2567 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 12 15:20:22.603947 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600334 2567 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600338 2567 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600341 2567 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600344 2567 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600347 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600350 2567 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600353 2567 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600356 2567 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600358 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600361 2567 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600364 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600366 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600369 2567 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600371 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600374 2567 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600376 2567 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600379 2567 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600382 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600384 2567 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 12 15:20:22.604467 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600387 2567 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600390 2567 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600392 2567 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600395 2567 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600397 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600400 2567 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600402 2567 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600405 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600407 2567 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600409 2567 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600412 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600414 2567 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600416 2567 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600419 2567 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600421 2567 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600423 2567 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600426 2567 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600429 2567 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600431 2567 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600434 2567 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 12 15:20:22.604945 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600437 2567 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600439 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600443 2567 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600446 2567 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600449 2567 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600451 2567 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600454 2567 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600457 2567 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600459 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600461 2567 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600464 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600466 2567 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600469 2567 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600472 2567 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600475 2567 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600478 2567 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600481 2567 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600484 2567 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600486 2567 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 12 15:20:22.605442 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600489 2567 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600492 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600495 2567 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600497 2567 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600500 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600503 2567 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600505 2567 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600508 2567 feature_gate.go:328] unrecognized feature gate: Example Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600510 2567 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600512 2567 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600515 2567 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600517 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600520 2567 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600523 2567 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600525 2567 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600527 2567 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600530 2567 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600533 2567 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600535 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600538 2567 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 12 15:20:22.605892 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600540 2567 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600543 2567 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600545 2567 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600548 2567 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600550 2567 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600552 2567 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600555 2567 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600557 2567 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600559 2567 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600562 2567 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600565 2567 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600567 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600569 2567 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600572 2567 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600574 2567 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600577 2567 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600579 2567 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600582 2567 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600584 2567 feature_gate.go:328] unrecognized feature gate: Example2 Jul 12 15:20:22.606379 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:22.600586 2567 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.600591 2567 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.600698 2567 server.go:962] "Client rotation is on, will bootstrap in background" Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.602755 2567 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.604074 2567 server.go:1019] "Starting client certificate rotation" Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.604180 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.604221 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 12 15:20:22.606831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.606666 2567 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 15:20:22.609280 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.609239 2567 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 12 15:20:22.621624 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.621601 2567 log.go:25] "Validated CRI v1 runtime API" Jul 12 15:20:22.623799 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.623782 2567 log.go:25] "Validated CRI v1 image API" Jul 12 15:20:22.625453 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.625427 2567 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 12 15:20:22.629554 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.629530 2567 fs.go:135] Filesystem UUIDs: map[72670ef8-3008-4d60-b718-7c0903f655f8:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 df6cba3f-17a8-4071-84ed-e6d05dc3ae1c:/dev/nvme0n1p3] Jul 12 15:20:22.629645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.629553 2567 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 12 15:20:22.634168 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.634146 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 15:20:22.634734 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.634620 2567 manager.go:216] Machine: {Timestamp:2026-07-12 15:20:22.633288464 +0000 UTC m=+0.342096048 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102213 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29791a817ffeb1751030937a5c2ee1 SystemUUID:ec29791a-817f-feb1-7510-30937a5c2ee1 BootID:dce85a0f-eda9-47c6-835d-7fcd3efd9c7c Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:34:97:95:4f:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:34:97:95:4f:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d2:6d:2e:07:c9:11 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 12 15:20:22.634836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.634735 2567 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 12 15:20:22.634890 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.634854 2567 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 12 15:20:22.638971 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.638942 2567 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 12 15:20:22.639166 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.638974 2567 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-136-30.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 12 15:20:22.639758 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.639746 2567 topology_manager.go:138] "Creating topology manager with none policy" Jul 12 15:20:22.639811 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.639765 2567 container_manager_linux.go:309] "Creating device plugin manager" Jul 12 15:20:22.639811 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.639776 2567 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 15:20:22.641124 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.641112 2567 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 12 15:20:22.643013 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.642989 2567 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 12 15:20:22.646361 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646347 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 12 15:20:22.646499 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646488 2567 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 12 15:20:22.646555 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646527 2567 kubelet.go:486] "Attempting to sync node with API server" Jul 12 15:20:22.646555 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646542 2567 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 12 15:20:22.646642 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646564 2567 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 12 15:20:22.646642 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646578 2567 kubelet.go:398] "Adding apiserver pod source" Jul 12 15:20:22.646642 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.646607 2567 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 12 15:20:22.647034 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.647020 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 15:20:22.647147 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.647134 2567 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 12 15:20:22.649782 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.649764 2567 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 12 15:20:22.652127 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652103 2567 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 12 15:20:22.652216 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652153 2567 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 12 15:20:22.652420 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652402 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 12 15:20:22.652518 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652435 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 12 15:20:22.652518 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652445 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 12 15:20:22.652518 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652453 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 12 15:20:22.652518 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652465 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 12 15:20:22.652518 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652475 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 12 15:20:22.652908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652893 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 12 15:20:22.652967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652912 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 12 15:20:22.652967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652926 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 12 15:20:22.652967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652935 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 12 15:20:22.652967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652955 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 12 15:20:22.652967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.652969 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 12 15:20:22.653719 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.653703 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 12 15:20:22.653810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.653731 2567 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 12 15:20:22.653810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.653727 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2jd7k" Jul 12 15:20:22.655835 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.655815 2567 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-136-30.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 12 15:20:22.656555 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.656530 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 12 15:20:22.656555 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.656536 2567 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-136-30.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 12 15:20:22.657265 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.657252 2567 server.go:1268] "Started kubelet" Jul 12 15:20:22.657399 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.657336 2567 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 12 15:20:22.657605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.657341 2567 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 12 15:20:22.657669 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.657633 2567 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 12 15:20:22.658130 ip-10-0-136-30 systemd[1]: Started Kubernetes Kubelet. Jul 12 15:20:22.659346 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.659330 2567 server.go:310] "Adding debug handlers to kubelet server" Jul 12 15:20:22.659346 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.659336 2567 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 12 15:20:22.661107 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.661088 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-2jd7k" Jul 12 15:20:22.662475 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.661584 2567 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-136-30.ec2.internal.18c193f870180d03 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-136-30.ec2.internal,UID:ip-10-0-136-30.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-136-30.ec2.internal,},FirstTimestamp:2026-07-12 15:20:22.657223939 +0000 UTC m=+0.366031525,LastTimestamp:2026-07-12 15:20:22.657223939 +0000 UTC m=+0.366031525,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-136-30.ec2.internal,}" Jul 12 15:20:22.663896 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.663869 2567 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 12 15:20:22.664781 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.664767 2567 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 12 15:20:22.664836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.664795 2567 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 12 15:20:22.664883 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.664867 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:22.665494 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665459 2567 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 12 15:20:22.665494 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665476 2567 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 12 15:20:22.665494 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665492 2567 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 12 15:20:22.665890 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665621 2567 reconstruct.go:97] "Volume reconstruction finished" Jul 12 15:20:22.665890 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665629 2567 reconciler.go:29] "Reconciler: start to sync state" Jul 12 15:20:22.665890 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665830 2567 factory.go:55] Registering systemd factory Jul 12 15:20:22.665890 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.665869 2567 factory.go:223] Registration of the systemd container factory successfully Jul 12 15:20:22.666107 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666098 2567 factory.go:153] Registering CRI-O factory Jul 12 15:20:22.666152 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666112 2567 factory.go:223] Registration of the crio container factory successfully Jul 12 15:20:22.666192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666158 2567 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 12 15:20:22.666192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666181 2567 factory.go:103] Registering Raw factory Jul 12 15:20:22.666283 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666196 2567 manager.go:1196] Started watching for new ooms in manager Jul 12 15:20:22.666877 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.666858 2567 manager.go:349] Starting recovery of all containers Jul 12 15:20:22.675336 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.675295 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:22.677324 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.677159 2567 manager.go:354] Recovery completed Jul 12 15:20:22.678159 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.678136 2567 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-136-30.ec2.internal\" not found" node="ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.682708 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.682696 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.684529 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.684516 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.684588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.684543 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.684588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.684554 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.685117 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685105 2567 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 12 15:20:22.685117 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685115 2567 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 12 15:20:22.685204 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685118 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.685204 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685130 2567 state_mem.go:36] "Initialized new in-memory state store" Jul 12 15:20:22.685749 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685733 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.685810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685764 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.685810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.685780 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.686382 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.686369 2567 policy_none.go:49] "None policy: Start" Jul 12 15:20:22.686435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.686385 2567 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 12 15:20:22.686435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.686396 2567 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 12 15:20:22.687705 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.687694 2567 policy_none.go:47] "Start" Jul 12 15:20:22.724634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.724611 2567 manager.go:337] "Starting Device Plugin manager" Jul 12 15:20:22.724742 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.724648 2567 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 12 15:20:22.724742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.724662 2567 server.go:87] "Starting device plugin registration server" Jul 12 15:20:22.724936 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.724923 2567 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 12 15:20:22.725015 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.724936 2567 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 12 15:20:22.725104 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.725028 2567 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 12 15:20:22.725152 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.725134 2567 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 12 15:20:22.725152 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.725143 2567 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 12 15:20:22.725692 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.725665 2567 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 12 15:20:22.725765 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.725721 2567 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:22.825652 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.825598 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.826800 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.826671 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.826800 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.826703 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.826800 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.826713 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.826800 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.826741 2567 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.834683 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.834639 2567 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.834683 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.834670 2567 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-136-30.ec2.internal\": node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:22.837867 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.837839 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 12 15:20:22.839271 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.839251 2567 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 12 15:20:22.839368 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.839288 2567 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 12 15:20:22.839368 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.839312 2567 kubelet.go:2448] "Starting kubelet main sync loop" Jul 12 15:20:22.839368 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.839351 2567 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 12 15:20:22.841586 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.841571 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:22.852552 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.852534 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:22.940065 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.940006 2567 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal"] Jul 12 15:20:22.940202 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.940134 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.941182 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.941157 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.941292 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.941189 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.941292 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.941198 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.942530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.942517 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.942642 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.942623 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.942686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.942652 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.943267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943242 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.943362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943275 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.943362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943288 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.943362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943299 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.943362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943321 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.943362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.943330 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.944527 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.944513 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.944571 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.944539 2567 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 12 15:20:22.945270 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.945253 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientMemory" Jul 12 15:20:22.945360 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.945284 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasNoDiskPressure" Jul 12 15:20:22.945360 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.945294 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeHasSufficientPID" Jul 12 15:20:22.953617 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.953600 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:22.967747 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.967721 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c6eb2977c26623da22c14690f0e26ab-config\") pod \"kube-apiserver-proxy-ip-10-0-136-30.ec2.internal\" (UID: \"5c6eb2977c26623da22c14690f0e26ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.967853 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.967752 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.967853 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:22.967771 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.967926 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.967887 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-30.ec2.internal\" not found" node="ip-10-0-136-30.ec2.internal" Jul 12 15:20:22.972459 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:22.972442 2567 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-136-30.ec2.internal\" not found" node="ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.053863 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.053827 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.068770 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068751 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.068907 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068783 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.068907 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068804 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.068907 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068831 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c6eb2977c26623da22c14690f0e26ab-config\") pod \"kube-apiserver-proxy-ip-10-0-136-30.ec2.internal\" (UID: \"5c6eb2977c26623da22c14690f0e26ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.068907 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068809 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/5c6eb2977c26623da22c14690f0e26ab-config\") pod \"kube-apiserver-proxy-ip-10-0-136-30.ec2.internal\" (UID: \"5c6eb2977c26623da22c14690f0e26ab\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.068907 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.068858 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fcfb6dd345873fb0734fae4ee5949856-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal\" (UID: \"fcfb6dd345873fb0734fae4ee5949856\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.154251 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.154186 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.254824 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.254806 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.270323 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.270288 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.276053 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.276018 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.357502 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.357448 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.458686 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.458602 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.565832 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.565797 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.604742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.604719 2567 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 12 15:20:23.605348 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.604909 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 15:20:23.605348 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.604918 2567 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 12 15:20:23.666107 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.666061 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.666107 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.666075 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-11 15:15:23 +0000 UTC" deadline="2028-04-09 03:33:04.555544348 +0000 UTC" Jul 12 15:20:23.666107 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.666110 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15276h12m40.889438621s" Jul 12 15:20:23.766226 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.766177 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.866951 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:23.866921 2567 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-136-30.ec2.internal\" not found" Jul 12 15:20:23.873450 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:23.873416 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcfb6dd345873fb0734fae4ee5949856.slice/crio-c80dd9f7a96ae173c2988632f9ff21ecc2c6e07623e0f24cf05aad95d6619612 WatchSource:0}: Error finding container c80dd9f7a96ae173c2988632f9ff21ecc2c6e07623e0f24cf05aad95d6619612: Status 404 returned error can't find the container with id c80dd9f7a96ae173c2988632f9ff21ecc2c6e07623e0f24cf05aad95d6619612 Jul 12 15:20:23.873857 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:23.873835 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c6eb2977c26623da22c14690f0e26ab.slice/crio-289050c934073a89ab0abb580455d93aadc5cb369fb8421a82535c92a0c4d95c WatchSource:0}: Error finding container 289050c934073a89ab0abb580455d93aadc5cb369fb8421a82535c92a0c4d95c: Status 404 returned error can't find the container with id 289050c934073a89ab0abb580455d93aadc5cb369fb8421a82535c92a0c4d95c Jul 12 15:20:23.878812 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.878798 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:20:23.888510 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.888492 2567 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:23.965410 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.965358 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.977617 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.977586 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 15:20:23.979741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.979720 2567 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" Jul 12 15:20:23.990382 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:23.990358 2567 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 12 15:20:24.170002 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.169967 2567 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:24.540837 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.540742 2567 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:24.649379 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.649346 2567 apiserver.go:52] "Watching apiserver" Jul 12 15:20:24.662948 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.662909 2567 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 12 15:20:24.665602 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.665574 2567 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 12 15:20:24.665742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.665638 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-7g55n","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal","openshift-multus/multus-jvjfn","openshift-multus/network-metrics-daemon-5x2w7","openshift-network-diagnostics/network-check-target-qftmv","openshift-network-operator/iptables-alerter-wgj8q","kube-system/konnectivity-agent-tnshv","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj","openshift-cluster-node-tuning-operator/tuned-qnx9s","openshift-dns/node-resolver-5xx64","openshift-multus/multus-additional-cni-plugins-hvncx","openshift-ovn-kubernetes/ovnkube-node-w686n","kube-system/global-pull-secret-syncer-bb2p6","kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal"] Jul 12 15:20:24.667478 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.667457 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.669904 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.669853 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.670023 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.669934 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:24.670337 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.670316 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 12 15:20:24.670595 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.670579 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-rnvk9\"" Jul 12 15:20:24.670853 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.670834 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 12 15:20:24.670935 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.670870 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.673355 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.673336 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.673881 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ltxg5\"" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.674111 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.674330 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.674506 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.674597 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.674777 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.674713 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:24.675176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.674900 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:24.675176 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.674973 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:24.677175 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.676238 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.677987 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.677690 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.678316 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.678292 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29a191cb-53c1-4bd2-accd-e848a25fa46e-agent-certs\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.678420 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.678329 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29a191cb-53c1-4bd2-accd-e848a25fa46e-konnectivity-ca\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.680362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.678963 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.680362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.679268 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-xsn9m\"" Jul 12 15:20:24.680362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.679522 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 12 15:20:24.680362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.679758 2567 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 12 15:20:24.680757 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.680376 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.680757 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.680642 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-97l65\"" Jul 12 15:20:24.680757 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.680693 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.681084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.681060 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.681523 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.681479 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.682581 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.682275 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.683046 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.683020 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 12 15:20:24.683637 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.683614 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-w8qnk\"" Jul 12 15:20:24.683743 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.683688 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.683743 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.683614 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.684394 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.684355 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.684878 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.684860 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.684977 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.684899 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 12 15:20:24.685174 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.685156 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-cfqkk\"" Jul 12 15:20:24.685251 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.685177 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.686010 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.685989 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.687531 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.687511 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.688391 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.688373 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.688485 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.688413 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-mpr54\"" Jul 12 15:20:24.689980 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.689962 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 12 15:20:24.690282 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690261 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 12 15:20:24.690391 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690202 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-9dvrq\"" Jul 12 15:20:24.690505 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690197 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.690753 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690731 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 12 15:20:24.690829 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690789 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 12 15:20:24.690869 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690852 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 12 15:20:24.690912 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.690736 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 12 15:20:24.691457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.691440 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4phfh\"" Jul 12 15:20:24.691615 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.691441 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 12 15:20:24.692157 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.692135 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 12 15:20:24.704241 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.704217 2567 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-pfkwr" Jul 12 15:20:24.713413 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.713382 2567 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-pfkwr" Jul 12 15:20:24.766954 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.766927 2567 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 12 15:20:24.779450 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779419 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-kubelet\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779470 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebdc7ad5-5993-472f-8dec-6d457d659cb6-host-slash\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-kubernetes\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779526 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkz94\" (UniqueName: \"kubernetes.io/projected/861bd4ff-c339-4c43-af8b-ae1e4790d18c-kube-api-access-vkz94\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779555 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779583 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.779622 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-tmp-dir\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779632 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-daemon-config\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrsxj\" (UniqueName: \"kubernetes.io/projected/ebdc7ad5-5993-472f-8dec-6d457d659cb6-kube-api-access-qrsxj\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779685 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-sys\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779710 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-os-release\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779735 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-kubelet\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779779 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovn-node-metrics-cert\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779847 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-device-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779891 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g94l\" (UniqueName: \"kubernetes.io/projected/c5e95be1-b01e-4b09-8f0f-765866af818c-kube-api-access-5g94l\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779921 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-system-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.779944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779948 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-node-log\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.779984 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-env-overrides\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780011 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-bin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780035 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-systemd-units\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780058 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-config\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-tuned\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780120 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec638ef4-7065-4b9a-9ac7-102a3ad10865-host\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780153 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780181 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q676x\" (UniqueName: \"kubernetes.io/projected/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-kube-api-access-q676x\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780212 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-kubelet-config\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780266 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncs7q\" (UniqueName: \"kubernetes.io/projected/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-kube-api-access-ncs7q\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780317 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-run\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780369 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-lib-modules\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780397 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-bin\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780424 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-hosts-file\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.780464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29a191cb-53c1-4bd2-accd-e848a25fa46e-agent-certs\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780483 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-os-release\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780522 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-netns\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780553 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-hostroot\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780578 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysconfig\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780606 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780633 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-socket-dir-parent\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780657 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-modprobe-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780683 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-conf\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-host\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-system-cni-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wjqn7\" (UniqueName: \"kubernetes.io/projected/627256e1-05a0-4ce5-afa6-9c582f6161bf-kube-api-access-wjqn7\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780783 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780819 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780852 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cnibin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780878 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cni-binary-copy\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780890 2567 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 12 15:20:24.781136 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780915 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-conf-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.780973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-etc-kubernetes\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781017 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebdc7ad5-5993-472f-8dec-6d457d659cb6-iptables-alerter-script\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781043 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-var-lib-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781093 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-log-socket\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781117 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-sys-fs\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781141 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781166 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-multus\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781191 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtbsq\" (UniqueName: \"kubernetes.io/projected/ec638ef4-7065-4b9a-9ac7-102a3ad10865-kube-api-access-wtbsq\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781218 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-script-lib\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781247 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781286 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpwpq\" (UniqueName: \"kubernetes.io/projected/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-kube-api-access-fpwpq\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-dbus\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-systemd\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781363 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-cnibin\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781389 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-slash\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.781898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781414 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-netns\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781443 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781469 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781508 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-k8s-cni-cncf-io\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781533 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-tmp\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781567 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781602 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29a191cb-53c1-4bd2-accd-e848a25fa46e-konnectivity-ca\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781628 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-var-lib-kubelet\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781655 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-systemd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-registration-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781706 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-multus-certs\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781732 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-ovn\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781759 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781786 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781825 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggzpg\" (UniqueName: \"kubernetes.io/projected/c18abc6a-0f18-4948-8c24-6bda26a89278-kube-api-access-ggzpg\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781856 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-socket-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.782844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781881 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec638ef4-7065-4b9a-9ac7-102a3ad10865-serviceca\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.783605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781911 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-etc-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.783605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781942 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.783605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.781974 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-netd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.783605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.782698 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/29a191cb-53c1-4bd2-accd-e848a25fa46e-konnectivity-ca\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.786348 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.786322 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/29a191cb-53c1-4bd2-accd-e848a25fa46e-agent-certs\") pod \"konnectivity-agent-tnshv\" (UID: \"29a191cb-53c1-4bd2-accd-e848a25fa46e\") " pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.845000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.844936 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" event={"ID":"5c6eb2977c26623da22c14690f0e26ab","Type":"ContainerStarted","Data":"289050c934073a89ab0abb580455d93aadc5cb369fb8421a82535c92a0c4d95c"} Jul 12 15:20:24.846161 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.846122 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" event={"ID":"fcfb6dd345873fb0734fae4ee5949856","Type":"ContainerStarted","Data":"c80dd9f7a96ae173c2988632f9ff21ecc2c6e07623e0f24cf05aad95d6619612"} Jul 12 15:20:24.882945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.882910 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-os-release\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.882945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.882947 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-netns\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.882945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.882964 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-hostroot\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.882983 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysconfig\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883009 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883036 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-socket-dir-parent\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883062 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-modprobe-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883086 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-netns\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883107 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-conf\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883090 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-hostroot\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883101 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-os-release\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883140 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-host\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883161 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysconfig\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883170 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-system-cni-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883165 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-socket-dir-parent\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883198 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wjqn7\" (UniqueName: \"kubernetes.io/projected/627256e1-05a0-4ce5-afa6-9c582f6161bf-kube-api-access-wjqn7\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883224 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883267 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883282 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-modprobe-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883293 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cnibin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cni-binary-copy\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883323 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-conf\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-conf-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883366 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-etc-kubernetes\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883376 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-host\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883395 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebdc7ad5-5993-472f-8dec-6d457d659cb6-iptables-alerter-script\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883420 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-var-lib-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883442 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-log-socket\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883458 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cnibin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883465 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-sys-fs\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883488 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883502 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-multus\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtbsq\" (UniqueName: \"kubernetes.io/projected/ec638ef4-7065-4b9a-9ac7-102a3ad10865-kube-api-access-wtbsq\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883545 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883420 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-system-cni-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.883915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-script-lib\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883619 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-etc-kubernetes\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpwpq\" (UniqueName: \"kubernetes.io/projected/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-kube-api-access-fpwpq\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883670 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-dbus\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883701 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-systemd\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883730 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-cnibin\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883765 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-var-lib-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883808 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-slash\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884082 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-systemd\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-conf-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884292 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-dbus\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884350 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884392 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-log-socket\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884402 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/ebdc7ad5-5993-472f-8dec-6d457d659cb6-iptables-alerter-script\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883765 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-slash\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884450 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-netns\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884442 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-sys-fs\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.884535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884505 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-cnibin\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884520 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-multus\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884460 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-cni-binary-copy\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.883718 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884555 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-k8s-cni-cncf-io\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884629 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-script-lib\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-tmp\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884700 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-netns\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884701 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-k8s-cni-cncf-io\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.884730 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.884782 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.884882 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:25.384856502 +0000 UTC m=+3.093664091 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.884919 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:25.384897829 +0000 UTC m=+3.093705409 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-var-lib-kubelet\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.885383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884983 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-systemd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.884999 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-etc-selinux\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-registration-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885043 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-multus-certs\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885085 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-systemd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885121 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-var-lib-kubelet\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885122 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-ovn\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885163 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-run-ovn\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885184 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-registration-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885200 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-run-multus-certs\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885194 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885244 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggzpg\" (UniqueName: \"kubernetes.io/projected/c18abc6a-0f18-4948-8c24-6bda26a89278-kube-api-access-ggzpg\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885274 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885298 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-socket-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885324 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec638ef4-7065-4b9a-9ac7-102a3ad10865-serviceca\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885352 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-etc-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885405 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-sysctl-d\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885398 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-netd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885440 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-kubelet\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885460 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-etc-openvswitch\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebdc7ad5-5993-472f-8dec-6d457d659cb6-host-slash\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885509 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-netd\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885512 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-kubernetes\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885521 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-kubelet\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885549 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vkz94\" (UniqueName: \"kubernetes.io/projected/861bd4ff-c339-4c43-af8b-ae1e4790d18c-kube-api-access-vkz94\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885580 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885568 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ebdc7ad5-5993-472f-8dec-6d457d659cb6-host-slash\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885554 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-kubernetes\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885690 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-socket-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885722 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885735 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885789 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-tmp-dir\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.886816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885805 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-run-ovn-kubernetes\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885816 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-daemon-config\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885840 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrsxj\" (UniqueName: \"kubernetes.io/projected/ebdc7ad5-5993-472f-8dec-6d457d659cb6-kube-api-access-qrsxj\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885863 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-sys\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-os-release\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885912 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-kubelet\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885935 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovn-node-metrics-cert\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885960 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-device-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.885986 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g94l\" (UniqueName: \"kubernetes.io/projected/c5e95be1-b01e-4b09-8f0f-765866af818c-kube-api-access-5g94l\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886011 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-system-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886033 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-node-log\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-env-overrides\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886112 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-bin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-tmp-dir\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886146 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-kubelet\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886213 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-sys\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886226 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-host-var-lib-cni-bin\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886276 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/627256e1-05a0-4ce5-afa6-9c582f6161bf-os-release\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.887479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-system-cni-dir\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.888280 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886307 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.888280 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886318 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/c5e95be1-b01e-4b09-8f0f-765866af818c-device-dir\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.888280 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886328 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-node-log\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.888280 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.886365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-systemd-units\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.888513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888444 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-config\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.888513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-tuned\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.888654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888551 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec638ef4-7065-4b9a-9ac7-102a3ad10865-host\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.888654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888578 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-tmp\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.888654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888588 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.888654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q676x\" (UniqueName: \"kubernetes.io/projected/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-kube-api-access-q676x\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888665 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-kubelet-config\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888690 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ncs7q\" (UniqueName: \"kubernetes.io/projected/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-kube-api-access-ncs7q\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888711 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-run\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888718 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ec638ef4-7065-4b9a-9ac7-102a3ad10865-serviceca\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888728 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-lib-modules\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888799 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-bin\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.888856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888816 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-lib-modules\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.889277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888888 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/861bd4ff-c339-4c43-af8b-ae1e4790d18c-run\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.889277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.888911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-hosts-file\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.889277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889098 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/627256e1-05a0-4ce5-afa6-9c582f6161bf-cni-binary-copy\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.889277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889213 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ec638ef4-7065-4b9a-9ac7-102a3ad10865-host\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.889277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889218 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-host-cni-bin\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.889533 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889312 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/4676e89d-0cdb-4fb5-9871-6daebefa4b95-kubelet-config\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:24.889780 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889691 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-hosts-file\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.893267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889894 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovnkube-config\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.893267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.889995 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-systemd-units\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.893267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.890014 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-multus-daemon-config\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.893267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.890253 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-env-overrides\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.893267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.891419 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/861bd4ff-c339-4c43-af8b-ae1e4790d18c-etc-tuned\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.895572 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.895547 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-ovn-node-metrics-cert\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.896252 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.896229 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:24.896397 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.896254 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:24.896397 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.896270 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:24.896397 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:24.896349 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:25.396329571 +0000 UTC m=+3.105137163 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:24.898946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.898919 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wjqn7\" (UniqueName: \"kubernetes.io/projected/627256e1-05a0-4ce5-afa6-9c582f6161bf-kube-api-access-wjqn7\") pod \"multus-additional-cni-plugins-hvncx\" (UID: \"627256e1-05a0-4ce5-afa6-9c582f6161bf\") " pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:24.899761 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.899730 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpwpq\" (UniqueName: \"kubernetes.io/projected/65cba0d8-d7a9-4d40-893b-b082b5f72ae4-kube-api-access-fpwpq\") pod \"node-resolver-5xx64\" (UID: \"65cba0d8-d7a9-4d40-893b-b082b5f72ae4\") " pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:24.900164 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.900141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtbsq\" (UniqueName: \"kubernetes.io/projected/ec638ef4-7065-4b9a-9ac7-102a3ad10865-kube-api-access-wtbsq\") pod \"node-ca-7g55n\" (UID: \"ec638ef4-7065-4b9a-9ac7-102a3ad10865\") " pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:24.904067 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.904019 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrsxj\" (UniqueName: \"kubernetes.io/projected/ebdc7ad5-5993-472f-8dec-6d457d659cb6-kube-api-access-qrsxj\") pod \"iptables-alerter-wgj8q\" (UID: \"ebdc7ad5-5993-472f-8dec-6d457d659cb6\") " pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:24.904263 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.904210 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q676x\" (UniqueName: \"kubernetes.io/projected/f57a6fd0-464d-4414-9593-5dfe4ffa93e5-kube-api-access-q676x\") pod \"ovnkube-node-w686n\" (UID: \"f57a6fd0-464d-4414-9593-5dfe4ffa93e5\") " pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:24.905736 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.905706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g94l\" (UniqueName: \"kubernetes.io/projected/c5e95be1-b01e-4b09-8f0f-765866af818c-kube-api-access-5g94l\") pod \"aws-ebs-csi-driver-node-sxhjj\" (UID: \"c5e95be1-b01e-4b09-8f0f-765866af818c\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:24.905809 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.905787 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncs7q\" (UniqueName: \"kubernetes.io/projected/66a7eaab-82df-44e4-8fd2-0b0cd83648f7-kube-api-access-ncs7q\") pod \"multus-jvjfn\" (UID: \"66a7eaab-82df-44e4-8fd2-0b0cd83648f7\") " pod="openshift-multus/multus-jvjfn" Jul 12 15:20:24.905845 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.905817 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkz94\" (UniqueName: \"kubernetes.io/projected/861bd4ff-c339-4c43-af8b-ae1e4790d18c-kube-api-access-vkz94\") pod \"tuned-qnx9s\" (UID: \"861bd4ff-c339-4c43-af8b-ae1e4790d18c\") " pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:24.907442 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.907415 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggzpg\" (UniqueName: \"kubernetes.io/projected/c18abc6a-0f18-4948-8c24-6bda26a89278-kube-api-access-ggzpg\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:24.984043 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.983996 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:24.994202 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:24.994176 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-jvjfn" Jul 12 15:20:25.009230 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.009204 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgj8q" Jul 12 15:20:25.013348 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.013326 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" Jul 12 15:20:25.024294 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.024263 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7g55n" Jul 12 15:20:25.033324 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.033299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" Jul 12 15:20:25.048450 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.048419 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5xx64" Jul 12 15:20:25.056583 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.056560 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-hvncx" Jul 12 15:20:25.063888 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.063864 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:25.084741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.084699 2567 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 12 15:20:25.393634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.393593 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:25.393853 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.393644 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:25.393853 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.393772 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:25.393853 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.393773 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:25.393853 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.393842 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:26.393821303 +0000 UTC m=+4.102628880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:25.394105 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.393863 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:26.393851291 +0000 UTC m=+4.102658868 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:25.495178 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.494316 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:25.495178 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.494530 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:25.495178 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.494550 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:25.495178 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.494564 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:25.495178 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.494625 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:26.494606946 +0000 UTC m=+4.203414521 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:25.716763 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.716651 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:15:25 +0000 UTC" deadline="2027-12-12 09:37:52.512896392 +0000 UTC" Jul 12 15:20:25.716763 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.716707 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12426h17m26.796194841s" Jul 12 15:20:25.839620 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.839584 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:25.839807 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:25.839632 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:25.839807 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.839712 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:25.839887 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:25.839832 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:25.907405 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.907375 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf57a6fd0_464d_4414_9593_5dfe4ffa93e5.slice/crio-63d85e324bcbe3473a1ebf6c02236bbf8fd976ff8af7f2d1c7dfabff7f34b8b2 WatchSource:0}: Error finding container 63d85e324bcbe3473a1ebf6c02236bbf8fd976ff8af7f2d1c7dfabff7f34b8b2: Status 404 returned error can't find the container with id 63d85e324bcbe3473a1ebf6c02236bbf8fd976ff8af7f2d1c7dfabff7f34b8b2 Jul 12 15:20:25.909015 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.908902 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5e95be1_b01e_4b09_8f0f_765866af818c.slice/crio-4e95f80398750dd9bbf848d01374b455ec215df1c2aa4fc27f174ac2e8dab04b WatchSource:0}: Error finding container 4e95f80398750dd9bbf848d01374b455ec215df1c2aa4fc27f174ac2e8dab04b: Status 404 returned error can't find the container with id 4e95f80398750dd9bbf848d01374b455ec215df1c2aa4fc27f174ac2e8dab04b Jul 12 15:20:25.911238 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.910639 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29a191cb_53c1_4bd2_accd_e848a25fa46e.slice/crio-081edfaa9548d39c920c141085778ff65a5bb6befb483dd505750a5828fb0ef0 WatchSource:0}: Error finding container 081edfaa9548d39c920c141085778ff65a5bb6befb483dd505750a5828fb0ef0: Status 404 returned error can't find the container with id 081edfaa9548d39c920c141085778ff65a5bb6befb483dd505750a5828fb0ef0 Jul 12 15:20:25.914151 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.914117 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podec638ef4_7065_4b9a_9ac7_102a3ad10865.slice/crio-f2d5b9675f39b41f5800b75b9fce21f98a583a77efd422f2767626d435f67f32 WatchSource:0}: Error finding container f2d5b9675f39b41f5800b75b9fce21f98a583a77efd422f2767626d435f67f32: Status 404 returned error can't find the container with id f2d5b9675f39b41f5800b75b9fce21f98a583a77efd422f2767626d435f67f32 Jul 12 15:20:25.914724 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.914628 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66a7eaab_82df_44e4_8fd2_0b0cd83648f7.slice/crio-a96bc80d2f9e4f59e8182d74fb2cfc70fe43bcde00dcc610284b393a01d2b120 WatchSource:0}: Error finding container a96bc80d2f9e4f59e8182d74fb2cfc70fe43bcde00dcc610284b393a01d2b120: Status 404 returned error can't find the container with id a96bc80d2f9e4f59e8182d74fb2cfc70fe43bcde00dcc610284b393a01d2b120 Jul 12 15:20:25.916370 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:25.916343 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65cba0d8_d7a9_4d40_893b_b082b5f72ae4.slice/crio-c73de88b8e0c01e6fd8174bdda73e6baf280a604b81785f5e3517f187532242d WatchSource:0}: Error finding container c73de88b8e0c01e6fd8174bdda73e6baf280a604b81785f5e3517f187532242d: Status 404 returned error can't find the container with id c73de88b8e0c01e6fd8174bdda73e6baf280a604b81785f5e3517f187532242d Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.401639 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.401938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.402087 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.402207 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:28.402182435 +0000 UTC m=+6.110990021 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.402639 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:26.402739 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.402698 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:28.402679282 +0000 UTC m=+6.111486858 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:26.504121 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.503404 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:26.504121 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.503600 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:26.504121 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.503625 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:26.504121 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.503638 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:26.504121 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.503706 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:28.503683617 +0000 UTC m=+6.212491192 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:26.717744 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.717510 2567 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-11 15:15:25 +0000 UTC" deadline="2028-02-14 11:12:34.453359334 +0000 UTC" Jul 12 15:20:26.717744 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.717554 2567 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13963h52m7.735811787s" Jul 12 15:20:26.843027 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.842508 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:26.843027 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:26.842653 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:26.859206 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.859168 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerStarted","Data":"3afb879e4c4eec8ef992d75ca5dee889cd40e60620fbf1fc984ee17f590c5e46"} Jul 12 15:20:26.867330 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.867284 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jvjfn" event={"ID":"66a7eaab-82df-44e4-8fd2-0b0cd83648f7","Type":"ContainerStarted","Data":"a96bc80d2f9e4f59e8182d74fb2cfc70fe43bcde00dcc610284b393a01d2b120"} Jul 12 15:20:26.872555 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.872518 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7g55n" event={"ID":"ec638ef4-7065-4b9a-9ac7-102a3ad10865","Type":"ContainerStarted","Data":"f2d5b9675f39b41f5800b75b9fce21f98a583a77efd422f2767626d435f67f32"} Jul 12 15:20:26.876416 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.876382 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"63d85e324bcbe3473a1ebf6c02236bbf8fd976ff8af7f2d1c7dfabff7f34b8b2"} Jul 12 15:20:26.892918 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.890786 2567 generic.go:356] "Generic (PLEG): container finished" podID="fcfb6dd345873fb0734fae4ee5949856" containerID="a0327e59fe31128d13e0e339bedf5e1acfdde2a8b29df75fc8ff16f218958556" exitCode=0 Jul 12 15:20:26.893193 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.890960 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" event={"ID":"fcfb6dd345873fb0734fae4ee5949856","Type":"ContainerDied","Data":"a0327e59fe31128d13e0e339bedf5e1acfdde2a8b29df75fc8ff16f218958556"} Jul 12 15:20:26.900516 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.900455 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgj8q" event={"ID":"ebdc7ad5-5993-472f-8dec-6d457d659cb6","Type":"ContainerStarted","Data":"e5e8fed027c96182e31a0b7bcf77dedd6826bf913fe8bc6c3788efaa41f11a2b"} Jul 12 15:20:26.908171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.902888 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5xx64" event={"ID":"65cba0d8-d7a9-4d40-893b-b082b5f72ae4","Type":"ContainerStarted","Data":"c73de88b8e0c01e6fd8174bdda73e6baf280a604b81785f5e3517f187532242d"} Jul 12 15:20:26.908171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.905891 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnshv" event={"ID":"29a191cb-53c1-4bd2-accd-e848a25fa46e","Type":"ContainerStarted","Data":"081edfaa9548d39c920c141085778ff65a5bb6befb483dd505750a5828fb0ef0"} Jul 12 15:20:26.931790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.931749 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" event={"ID":"c5e95be1-b01e-4b09-8f0f-765866af818c","Type":"ContainerStarted","Data":"4e95f80398750dd9bbf848d01374b455ec215df1c2aa4fc27f174ac2e8dab04b"} Jul 12 15:20:26.935457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.935417 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" event={"ID":"5c6eb2977c26623da22c14690f0e26ab","Type":"ContainerStarted","Data":"91863e78668f79fb4ac9f5134be14e55aac0a17ed6d4a5379b67fb1147150c82"} Jul 12 15:20:26.946372 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.946331 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" event={"ID":"861bd4ff-c339-4c43-af8b-ae1e4790d18c","Type":"ContainerStarted","Data":"46ad73169fcf3e126d578bd2afb1bb286d2903f8304f753c2d98d2bc40ea7d82"} Jul 12 15:20:26.951837 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:26.951767 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-136-30.ec2.internal" podStartSLOduration=2.951745682 podStartE2EDuration="2.951745682s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:26.951364092 +0000 UTC m=+4.660171693" watchObservedRunningTime="2026-07-12 15:20:26.951745682 +0000 UTC m=+4.660553280" Jul 12 15:20:27.839919 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:27.839879 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:27.840513 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:27.840035 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:27.840513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:27.840498 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:27.840686 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:27.840614 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:27.958278 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:27.958195 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" event={"ID":"fcfb6dd345873fb0734fae4ee5949856","Type":"ContainerStarted","Data":"15077a303446fec28ac9f0f1042c21c1f93312f8f77c7feaed291bbd6552e020"} Jul 12 15:20:28.421381 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:28.421321 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:28.421381 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:28.421376 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:28.421660 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.421543 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:28.421660 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.421612 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:32.42159147 +0000 UTC m=+10.130399043 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:28.422041 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.422017 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:28.422148 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.422080 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:32.422063891 +0000 UTC m=+10.130871467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:28.522534 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:28.522490 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:28.522764 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.522737 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:28.522764 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.522766 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:28.522891 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.522780 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:28.522891 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.522842 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:32.522822455 +0000 UTC m=+10.231630031 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:28.842544 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:28.842507 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:28.843034 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:28.842684 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:29.840572 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:29.840539 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:29.840773 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:29.840667 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:29.840773 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:29.840725 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:29.840899 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:29.840794 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:30.840342 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:30.840299 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:30.840734 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:30.840540 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:31.839657 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:31.839600 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:31.839865 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:31.839609 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:31.839865 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:31.839763 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:31.839865 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:31.839833 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:32.463499 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:32.463455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:32.463973 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:32.463513 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:32.463973 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.463645 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:32.463973 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.463712 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:40.463692915 +0000 UTC m=+18.172500493 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:32.464225 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.464164 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:32.464225 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.464217 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:40.464200599 +0000 UTC m=+18.173008176 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:32.564689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:32.564648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:32.564884 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.564863 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:32.564945 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.564887 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:32.564945 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.564901 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:32.565051 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.564966 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:40.564946677 +0000 UTC m=+18.273754273 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:32.840832 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:32.840784 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:32.841044 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:32.840930 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:33.840935 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:33.840380 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:33.840935 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:33.840501 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:33.840935 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:33.840813 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:33.840935 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:33.840881 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:34.841560 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:34.840981 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:34.841560 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:34.841153 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:35.840409 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:35.840366 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:35.840603 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:35.840417 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:35.840603 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:35.840530 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:35.840745 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:35.840696 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:36.840626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:36.840595 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:36.841029 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:36.840705 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:37.840126 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:37.840091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:37.840317 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:37.840091 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:37.840317 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:37.840220 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:37.840317 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:37.840312 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:38.840394 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:38.840352 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:38.840861 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:38.840491 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:39.840354 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:39.840317 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:39.840531 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:39.840451 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:39.840531 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:39.840516 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:39.840937 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:39.840597 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:40.531169 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:40.531123 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:40.531353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:40.531183 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:40.531353 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.531271 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:40.531353 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.531333 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:56.531317539 +0000 UTC m=+34.240125111 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:40.531520 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.531357 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:40.531520 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.531433 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:20:56.53141448 +0000 UTC m=+34.240222066 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:40.632440 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:40.632401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:40.632600 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.632534 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:40.632600 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.632550 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:40.632600 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.632558 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:40.632739 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.632626 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:20:56.632612645 +0000 UTC m=+34.341420230 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:40.840140 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:40.840109 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:40.840298 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:40.840220 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:41.840005 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:41.839964 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:41.840488 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:41.840114 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:41.840488 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:41.840182 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:41.840488 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:41.840268 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:42.844281 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:42.844246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:42.844649 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:42.844361 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:43.839610 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:43.839575 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:43.839610 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:43.839599 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:43.839836 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:43.839692 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:43.839836 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:43.839812 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:44.840347 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:44.840302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:44.840939 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:44.840451 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:45.840640 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:45.840390 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:45.841158 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:45.840461 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:45.841158 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:45.840750 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:45.841158 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:45.840783 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:46.001142 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.001109 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" event={"ID":"861bd4ff-c339-4c43-af8b-ae1e4790d18c","Type":"ContainerStarted","Data":"672cff98e2b9286866c73d91a4b218598e013aa42415dc94d400d0a022c13ee4"} Jul 12 15:20:46.002875 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.002845 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="2baacdbe7728457eb4a992fbe964bd9222218a98c546479939b03b7066d0e911" exitCode=0 Jul 12 15:20:46.003078 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.002940 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"2baacdbe7728457eb4a992fbe964bd9222218a98c546479939b03b7066d0e911"} Jul 12 15:20:46.004788 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.004763 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-jvjfn" event={"ID":"66a7eaab-82df-44e4-8fd2-0b0cd83648f7","Type":"ContainerStarted","Data":"ef779d3aebe41308117a00c910a5df87d28500c6b39ce67209f31637c65d3771"} Jul 12 15:20:46.006537 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.006323 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7g55n" event={"ID":"ec638ef4-7065-4b9a-9ac7-102a3ad10865","Type":"ContainerStarted","Data":"2c4b998b40d0eb79c4ee1034afb23affdb8f663e08b2c7d3e5c2385e4be636e9"} Jul 12 15:20:46.008621 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.008589 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:20:46.009022 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.008998 2567 generic.go:356] "Generic (PLEG): container finished" podID="f57a6fd0-464d-4414-9593-5dfe4ffa93e5" containerID="bf3a9553c177bf55141928d76b5cf3b28e87257bb4907e560a55a576f111a583" exitCode=1 Jul 12 15:20:46.009134 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.009026 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"ffa3b5f694d86daa97d1b7c33c52a2feead6731d1749d4899678c536e15682bd"} Jul 12 15:20:46.009335 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.009314 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerDied","Data":"bf3a9553c177bf55141928d76b5cf3b28e87257bb4907e560a55a576f111a583"} Jul 12 15:20:46.010319 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.010290 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"32f6295a2ed5d299e941332be7e971a08f59e25b7e9df188c63fbb2811e3e23a"} Jul 12 15:20:46.010592 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.010574 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5xx64" event={"ID":"65cba0d8-d7a9-4d40-893b-b082b5f72ae4","Type":"ContainerStarted","Data":"cd4f9ef5dd36ef6407e76efb99be89ee19ffbe4befcca0b29fabb3afcd9476dc"} Jul 12 15:20:46.011947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.011927 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tnshv" event={"ID":"29a191cb-53c1-4bd2-accd-e848a25fa46e","Type":"ContainerStarted","Data":"712e4619dd2a26f21a32b21289b3d8163aa1da6c58b6f7323f1a76b1f81fb728"} Jul 12 15:20:46.013413 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.013395 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" event={"ID":"c5e95be1-b01e-4b09-8f0f-765866af818c","Type":"ContainerStarted","Data":"2e5fa651c0336cbabc5dc401b66fe0a712966d7148804f2df1f533753a0b7e8d"} Jul 12 15:20:46.016747 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.016699 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-qnx9s" podStartSLOduration=3.461600266 podStartE2EDuration="23.016684664s" totalImagesPullingTime="19.555084398s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:46.016528809 +0000 UTC m=+23.725336405" watchObservedRunningTime="2026-07-12 15:20:46.016684664 +0000 UTC m=+23.725492259" Jul 12 15:20:46.017402 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.017284 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-136-30.ec2.internal" podStartSLOduration=22.017274241 podStartE2EDuration="22.017274241s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:27.97564284 +0000 UTC m=+5.684450457" watchObservedRunningTime="2026-07-12 15:20:46.017274241 +0000 UTC m=+23.726081836" Jul 12 15:20:46.029951 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.029905 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-5xx64" podStartSLOduration=3.473451428 podStartE2EDuration="23.02989085s" totalImagesPullingTime="19.556439422s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:46.029386149 +0000 UTC m=+23.738193744" watchObservedRunningTime="2026-07-12 15:20:46.02989085 +0000 UTC m=+23.738698444" Jul 12 15:20:46.042431 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.042388 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7g55n" podStartSLOduration=3.482185408 podStartE2EDuration="23.042372793s" totalImagesPullingTime="19.560187385s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:46.042362207 +0000 UTC m=+23.751169802" watchObservedRunningTime="2026-07-12 15:20:46.042372793 +0000 UTC m=+23.751180387" Jul 12 15:20:46.084904 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.084849 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tnshv" podStartSLOduration=3.5223508629999998 podStartE2EDuration="23.08482617s" totalImagesPullingTime="19.562475307s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:46.084300961 +0000 UTC m=+23.793108556" watchObservedRunningTime="2026-07-12 15:20:46.08482617 +0000 UTC m=+23.793633765" Jul 12 15:20:46.109669 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.109619 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-jvjfn" podStartSLOduration=3.490907249 podStartE2EDuration="23.109603072s" totalImagesPullingTime="19.618695823s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:46.108948799 +0000 UTC m=+23.817756390" watchObservedRunningTime="2026-07-12 15:20:46.109603072 +0000 UTC m=+23.818410685" Jul 12 15:20:46.843595 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:46.843302 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:46.844249 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:46.843648 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:47.018989 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.018951 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:20:47.019449 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.019416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"cd294ac4ee875e3944aa4d47c70393846d17039229fc805ae50af26ac0f0d88b"} Jul 12 15:20:47.019548 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.019511 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"8577602fbf8330a711066912b6ad73d73f28bdfe363b831c026420b9c8975acb"} Jul 12 15:20:47.019605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.019563 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"cfbe060a945c4ddb370099ade192b6272ff9fb2a904afa6e7349c5908127f8d1"} Jul 12 15:20:47.021034 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.021007 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgj8q" event={"ID":"ebdc7ad5-5993-472f-8dec-6d457d659cb6","Type":"ContainerStarted","Data":"3515ffd83d49ae889e5632fd13a3d1195b8febdedb6b2bd98cdf3a7ee4a07d33"} Jul 12 15:20:47.036735 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.036675 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wgj8q" podStartSLOduration=4.424476977 podStartE2EDuration="24.036656481s" totalImagesPullingTime="19.612179504s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:47.036247353 +0000 UTC m=+24.745054972" watchObservedRunningTime="2026-07-12 15:20:47.036656481 +0000 UTC m=+24.745464076" Jul 12 15:20:47.288334 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.288303 2567 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 12 15:20:47.738543 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.738447 2567 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-12T15:20:47.288332559Z","UUID":"4685b889-0953-4666-afa6-5837570286c6","Handler":null,"Name":"","Endpoint":""} Jul 12 15:20:47.740240 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.740217 2567 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 12 15:20:47.740375 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.740267 2567 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 12 15:20:47.840291 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.840259 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:47.840474 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:47.840372 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:47.840474 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:47.840408 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:47.840583 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:47.840482 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:48.026338 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:48.026199 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" event={"ID":"c5e95be1-b01e-4b09-8f0f-765866af818c","Type":"ContainerStarted","Data":"a2585270cbb6e2474a169de6fcf8ff2daa1f0ab516d74b495c553e5e87f51c9b"} Jul 12 15:20:48.839805 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:48.839769 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:48.839998 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:48.839904 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:49.034639 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.034602 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:20:49.035113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.035076 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"4c7bcf6843ec4574e8442b3ce60b639174b2351355ce48ad2f4c9b6d71586477"} Jul 12 15:20:49.037326 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.037298 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" event={"ID":"c5e95be1-b01e-4b09-8f0f-765866af818c","Type":"ContainerStarted","Data":"93e614048ec478d3d67163d65196c18ce5a873f345cb2bae714a70fccc8b3120"} Jul 12 15:20:49.054730 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.054675 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sxhjj" podStartSLOduration=4.187246717 podStartE2EDuration="26.054659771s" totalImagesPullingTime="21.867413054s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:49.054468335 +0000 UTC m=+26.763275928" watchObservedRunningTime="2026-07-12 15:20:49.054659771 +0000 UTC m=+26.763467364" Jul 12 15:20:49.840118 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.839888 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:49.840302 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:49.839939 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:49.840302 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:49.840228 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:49.840398 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:49.840315 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:50.607092 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:50.607064 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:50.608307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:50.608279 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:50.840370 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:50.840163 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:50.840512 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:50.840468 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:51.044852 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.044829 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:20:51.045245 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.045220 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"be3fd5537712055906681f7e292b31ec8d8f0be6770759c35603429fe3770238"} Jul 12 15:20:51.045436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.045414 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:51.045590 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.045499 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:51.045646 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.045635 2567 scope.go:117] "RemoveContainer" containerID="bf3a9553c177bf55141928d76b5cf3b28e87257bb4907e560a55a576f111a583" Jul 12 15:20:51.047424 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.047403 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="b86d1596f67f61729635612a95320b97a7c743dd8d44e52fff4e71dbc472d97a" exitCode=0 Jul 12 15:20:51.047534 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.047488 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"b86d1596f67f61729635612a95320b97a7c743dd8d44e52fff4e71dbc472d97a"} Jul 12 15:20:51.047745 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.047723 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:51.048223 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.048136 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tnshv" Jul 12 15:20:51.063064 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.063020 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:51.840238 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.840199 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:51.840644 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:51.840343 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:51.840644 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:51.840424 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:51.840644 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:51.840491 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:52.053806 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.053785 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:20:52.054216 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.054189 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" event={"ID":"f57a6fd0-464d-4414-9593-5dfe4ffa93e5","Type":"ContainerStarted","Data":"121dee8d9c0c815c6344163879097b39fa942f08f50fc04f8d608acbd0a5f1d7"} Jul 12 15:20:52.054582 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.054560 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:52.056357 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.056334 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="282190f0237e071f6e6ee069db1cdef0c1cd19fcacb6f5b0579191b01faebe0f" exitCode=0 Jul 12 15:20:52.056441 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.056395 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"282190f0237e071f6e6ee069db1cdef0c1cd19fcacb6f5b0579191b01faebe0f"} Jul 12 15:20:52.071014 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.070979 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:20:52.085656 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.085611 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" podStartSLOduration=9.415567977 podStartE2EDuration="29.085595223s" totalImagesPullingTime="19.670027246s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:20:52.084036726 +0000 UTC m=+29.792844333" watchObservedRunningTime="2026-07-12 15:20:52.085595223 +0000 UTC m=+29.794402817" Jul 12 15:20:52.485172 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.485116 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bb2p6"] Jul 12 15:20:52.485325 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.485310 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:52.485455 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:52.485428 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:52.488298 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.488271 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5x2w7"] Jul 12 15:20:52.488426 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.488390 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:52.488530 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:52.488508 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:52.488929 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.488902 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qftmv"] Jul 12 15:20:52.489036 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:52.488990 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:52.489139 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:52.489089 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:53.061748 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:53.061487 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="998de3d0cec9a28693d66732c49ac0bd67283ebb2972708212e3237201950cad" exitCode=0 Jul 12 15:20:53.062171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:53.061565 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"998de3d0cec9a28693d66732c49ac0bd67283ebb2972708212e3237201950cad"} Jul 12 15:20:53.840405 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:53.840369 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:53.840405 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:53.840393 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:53.840675 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:53.840504 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:53.840675 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:53.840643 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:54.840465 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:54.840383 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:54.840885 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:54.840517 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:55.840176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:55.840148 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:55.840340 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:55.840247 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:55.840340 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:55.840312 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:55.840417 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:55.840401 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:56.551429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:56.551385 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:56.552293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:56.551438 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:56.552293 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.551566 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:56.552293 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.551601 2567 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:56.552293 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.551643 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:28.551621247 +0000 UTC m=+66.260428838 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 12 15:20:56.552293 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.551666 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret podName:4676e89d-0cdb-4fb5-9871-6daebefa4b95 nodeName:}" failed. No retries permitted until 2026-07-12 15:21:28.55165597 +0000 UTC m=+66.260463546 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret") pod "global-pull-secret-syncer-bb2p6" (UID: "4676e89d-0cdb-4fb5-9871-6daebefa4b95") : object "kube-system"/"original-pull-secret" not registered Jul 12 15:20:56.652552 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:56.652514 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:56.652756 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.652675 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 12 15:20:56.652756 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.652704 2567 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 12 15:20:56.652756 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.652719 2567 projected.go:196] Error preparing data for projected volume kube-api-access-c9jss for pod openshift-network-diagnostics/network-check-target-qftmv: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:56.652928 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.652784 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss podName:f94feb18-159f-4512-a000-61cef68326cc nodeName:}" failed. No retries permitted until 2026-07-12 15:21:28.652763641 +0000 UTC m=+66.361571233 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-c9jss" (UniqueName: "kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss") pod "network-check-target-qftmv" (UID: "f94feb18-159f-4512-a000-61cef68326cc") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 12 15:20:56.839647 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:56.839611 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:56.839855 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:56.839806 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:20:57.840010 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:57.839974 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:57.840504 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:57.839974 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:57.840504 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:57.840094 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-qftmv" podUID="f94feb18-159f-4512-a000-61cef68326cc" Jul 12 15:20:57.840504 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:57.840163 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-bb2p6" podUID="4676e89d-0cdb-4fb5-9871-6daebefa4b95" Jul 12 15:20:58.081956 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.081917 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="5142ef92c27db2fed329e05ab4cdfa9db7e561bba5eea896e88e0287891146c6" exitCode=0 Jul 12 15:20:58.082169 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.081968 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"5142ef92c27db2fed329e05ab4cdfa9db7e561bba5eea896e88e0287891146c6"} Jul 12 15:20:58.576701 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.576671 2567 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-136-30.ec2.internal" event="NodeReady" Jul 12 15:20:58.576861 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.576783 2567 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 12 15:20:58.617025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.616990 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5xzvm"] Jul 12 15:20:58.641099 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.641067 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-g5ds8"] Jul 12 15:20:58.641254 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.641147 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.643689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.643655 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 12 15:20:58.643823 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.643751 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 12 15:20:58.643919 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.643896 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ns4rd\"" Jul 12 15:20:58.665015 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.664982 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5xzvm"] Jul 12 15:20:58.665015 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.665017 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5ds8"] Jul 12 15:20:58.665249 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.665149 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.667807 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.667786 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 12 15:20:58.667920 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.667820 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-79jmw\"" Jul 12 15:20:58.667974 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.667939 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 12 15:20:58.668039 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.668011 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 12 15:20:58.767528 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767487 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gv46\" (UniqueName: \"kubernetes.io/projected/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-kube-api-access-4gv46\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.767716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767546 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-config-volume\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.767716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767589 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.767716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767609 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-tmp-dir\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.767716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767624 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-spjzn\" (UniqueName: \"kubernetes.io/projected/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-kube-api-access-spjzn\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.767716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.767681 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-cert\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.839908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.839823 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:20:58.843852 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.843828 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 15:20:58.843852 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.843837 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ztf2\"" Jul 12 15:20:58.868156 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868119 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-config-volume\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.868293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868172 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.868293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868193 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-tmp-dir\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.868293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868216 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-spjzn\" (UniqueName: \"kubernetes.io/projected/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-kube-api-access-spjzn\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.868293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868240 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-cert\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.868293 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868260 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gv46\" (UniqueName: \"kubernetes.io/projected/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-kube-api-access-4gv46\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.868617 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:58.868410 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:58.868617 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:58.868487 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:20:59.368466607 +0000 UTC m=+37.077274208 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:20:58.868617 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-tmp-dir\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.868736 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.868719 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-config-volume\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.872663 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.872638 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-cert\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.877162 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.876935 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-spjzn\" (UniqueName: \"kubernetes.io/projected/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-kube-api-access-spjzn\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:58.877265 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.877069 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gv46\" (UniqueName: \"kubernetes.io/projected/1d63f4ab-9cfd-46d1-9dc5-0da56873da41-kube-api-access-4gv46\") pod \"ingress-canary-g5ds8\" (UID: \"1d63f4ab-9cfd-46d1-9dc5-0da56873da41\") " pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:58.973141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:58.973104 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-g5ds8" Jul 12 15:20:59.089638 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.089604 2567 generic.go:356] "Generic (PLEG): container finished" podID="627256e1-05a0-4ce5-afa6-9c582f6161bf" containerID="ce980609c8ad010c572ba33883873aa63761fcf6b55a96b3d0ead93cad133457" exitCode=0 Jul 12 15:20:59.089795 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.089671 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerDied","Data":"ce980609c8ad010c572ba33883873aa63761fcf6b55a96b3d0ead93cad133457"} Jul 12 15:20:59.164225 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.164191 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-g5ds8"] Jul 12 15:20:59.168262 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:20:59.168230 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d63f4ab_9cfd_46d1_9dc5_0da56873da41.slice/crio-5f397888ea7a9b812a33f7290bddc84f1d64bf04506a6a286f58ac160e6f585d WatchSource:0}: Error finding container 5f397888ea7a9b812a33f7290bddc84f1d64bf04506a6a286f58ac160e6f585d: Status 404 returned error can't find the container with id 5f397888ea7a9b812a33f7290bddc84f1d64bf04506a6a286f58ac160e6f585d Jul 12 15:20:59.372000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.371901 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:20:59.372180 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:59.372030 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:20:59.372180 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:20:59.372110 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:21:00.372092856 +0000 UTC m=+38.080900450 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:20:59.840301 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.840265 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:20:59.840492 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.840434 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:20:59.843228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.843192 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 15:20:59.843228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.843225 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 15:20:59.844369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.844351 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bw5ll\"" Jul 12 15:20:59.844790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:20:59.844408 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 15:21:00.094074 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:00.093965 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5ds8" event={"ID":"1d63f4ab-9cfd-46d1-9dc5-0da56873da41","Type":"ContainerStarted","Data":"5f397888ea7a9b812a33f7290bddc84f1d64bf04506a6a286f58ac160e6f585d"} Jul 12 15:21:00.097747 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:00.097717 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-hvncx" event={"ID":"627256e1-05a0-4ce5-afa6-9c582f6161bf","Type":"ContainerStarted","Data":"f5ac10533817caee0f246023b0c4ff8c6e88de119dba066eb16c96274cb974f2"} Jul 12 15:21:00.120315 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:00.120255 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-hvncx" podStartSLOduration=7.215209386 podStartE2EDuration="37.120236129s" totalImagesPullingTime="29.905026743s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:00.11842651 +0000 UTC m=+37.827234095" watchObservedRunningTime="2026-07-12 15:21:00.120236129 +0000 UTC m=+37.829043722" Jul 12 15:21:00.381771 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:00.381675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:21:00.381930 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:00.381807 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:00.381930 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:00.381866 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:21:02.381851297 +0000 UTC m=+40.090658882 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:21:02.104669 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:02.104420 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-g5ds8" event={"ID":"1d63f4ab-9cfd-46d1-9dc5-0da56873da41","Type":"ContainerStarted","Data":"1352f71d633419d9e7eb56074217ba8bb24ce0831a82e5bcb5e9b91f4216a7e5"} Jul 12 15:21:02.120982 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:02.120935 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-g5ds8" podStartSLOduration=1.461393111 podStartE2EDuration="4.120919473s" totalImagesPullingTime="2.659526362s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:02.12038244 +0000 UTC m=+39.829190035" watchObservedRunningTime="2026-07-12 15:21:02.120919473 +0000 UTC m=+39.829727066" Jul 12 15:21:02.396466 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:02.396383 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:21:02.396626 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:02.396513 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:02.396626 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:02.396570 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:21:06.396554192 +0000 UTC m=+44.105361784 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:21:06.420497 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:06.420452 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:21:06.421068 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:06.420595 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:06.421068 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:06.420656 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:21:14.42063895 +0000 UTC m=+52.129446536 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:21:14.472155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:14.472110 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:21:14.472635 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:14.472239 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:14.472635 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:14.472292 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:21:30.472277704 +0000 UTC m=+68.181085276 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:21:24.080076 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:24.080033 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-w686n" Jul 12 15:21:28.571941 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.571895 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:21:28.571941 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.571938 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:21:28.574498 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.574479 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 12 15:21:28.574553 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.574502 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 12 15:21:28.582622 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:28.582593 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:21:28.582700 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:28.582689 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:22:32.582666493 +0000 UTC m=+130.291474067 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : secret "metrics-daemon-secret" not found Jul 12 15:21:28.586568 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.586549 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/4676e89d-0cdb-4fb5-9871-6daebefa4b95-original-pull-secret\") pod \"global-pull-secret-syncer-bb2p6\" (UID: \"4676e89d-0cdb-4fb5-9871-6daebefa4b95\") " pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:21:28.652612 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.652577 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-bb2p6" Jul 12 15:21:28.673164 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.673133 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:21:28.675965 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.675946 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 12 15:21:28.686476 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.686454 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 12 15:21:28.697098 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.697071 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c9jss\" (UniqueName: \"kubernetes.io/projected/f94feb18-159f-4512-a000-61cef68326cc-kube-api-access-c9jss\") pod \"network-check-target-qftmv\" (UID: \"f94feb18-159f-4512-a000-61cef68326cc\") " pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:21:28.800949 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.800917 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-bb2p6"] Jul 12 15:21:28.960757 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.960667 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bw5ll\"" Jul 12 15:21:28.968840 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:28.968813 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:21:29.116132 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:29.116093 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-qftmv"] Jul 12 15:21:29.119789 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:21:29.119765 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf94feb18_159f_4512_a000_61cef68326cc.slice/crio-0078655f1b847045bc6ac229cd9ab02d6b34a226538f818c21d57d5e03eebf38 WatchSource:0}: Error finding container 0078655f1b847045bc6ac229cd9ab02d6b34a226538f818c21d57d5e03eebf38: Status 404 returned error can't find the container with id 0078655f1b847045bc6ac229cd9ab02d6b34a226538f818c21d57d5e03eebf38 Jul 12 15:21:29.163954 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:29.163917 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qftmv" event={"ID":"f94feb18-159f-4512-a000-61cef68326cc","Type":"ContainerStarted","Data":"0078655f1b847045bc6ac229cd9ab02d6b34a226538f818c21d57d5e03eebf38"} Jul 12 15:21:29.165002 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:29.164973 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bb2p6" event={"ID":"4676e89d-0cdb-4fb5-9871-6daebefa4b95","Type":"ContainerStarted","Data":"60be000ea9ad75acde9c38a603d10cac06638e396dcfcf3957333f1c364413a5"} Jul 12 15:21:30.486515 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:30.486470 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:21:30.487007 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:30.486593 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:21:30.487007 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:21:30.486644 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:22:02.486628924 +0000 UTC m=+100.195436497 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:21:34.178823 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:34.178781 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-qftmv" event={"ID":"f94feb18-159f-4512-a000-61cef68326cc","Type":"ContainerStarted","Data":"6df04236c4df6b885f48b360a072b0527e1636d004d52f771264e2ad2a96f15a"} Jul 12 15:21:34.179262 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:34.178884 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:21:34.180313 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:34.180291 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-bb2p6" event={"ID":"4676e89d-0cdb-4fb5-9871-6daebefa4b95","Type":"ContainerStarted","Data":"c1d9f29b7af60c05ac19c66c105f4197b83d4e7c7de2dcfab1855960bfc5bed8"} Jul 12 15:21:34.193235 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:34.193188 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-qftmv" podStartSLOduration=66.967529995 podStartE2EDuration="1m11.193172792s" totalImagesPullingTime="4.225642797s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:34.192683072 +0000 UTC m=+71.901490684" watchObservedRunningTime="2026-07-12 15:21:34.193172792 +0000 UTC m=+71.901980381" Jul 12 15:21:34.206223 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:21:34.206181 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-bb2p6" podStartSLOduration=66.663037055 podStartE2EDuration="1m11.206167157s" totalImagesPullingTime="4.543130102s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:21:34.20573952 +0000 UTC m=+71.914547091" watchObservedRunningTime="2026-07-12 15:21:34.206167157 +0000 UTC m=+71.914974751" Jul 12 15:22:02.504370 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:22:02.504311 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:22:02.504804 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:22:02.504467 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:22:02.504804 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:22:02.504537 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:23:06.504520183 +0000 UTC m=+164.213327755 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:22:05.185145 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:22:05.185112 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-qftmv" Jul 12 15:22:32.613638 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:22:32.613597 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:22:32.614170 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:22:32.613710 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:22:32.614170 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:22:32.613783 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:34.613759086 +0000 UTC m=+252.322566659 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : secret "metrics-daemon-secret" not found Jul 12 15:23:01.649683 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:23:01.649642 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5xzvm" podUID="3a635a9e-3dbc-40c4-b4ca-7240719f4e5c" Jul 12 15:23:01.848243 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:23:01.848199 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:23:02.392389 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:23:02.392365 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:23:06.549742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:23:06.549692 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:23:06.550200 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:23:06.549854 2567 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 12 15:23:06.550200 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:23:06.549927 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls podName:3a635a9e-3dbc-40c4-b4ca-7240719f4e5c nodeName:}" failed. No retries permitted until 2026-07-12 15:25:08.549907879 +0000 UTC m=+286.258715474 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls") pod "dns-default-5xzvm" (UID: "3a635a9e-3dbc-40c4-b4ca-7240719f4e5c") : secret "dns-default-metrics-tls" not found Jul 12 15:23:13.839773 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:23:13.839677 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:24:16.962519 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.962486 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-lk5sx"] Jul 12 15:24:16.964240 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.964223 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7"] Jul 12 15:24:16.964356 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.964338 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:16.965905 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.965884 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:16.967794 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.967775 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 12 15:24:16.967906 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.967844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 12 15:24:16.968512 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.968496 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-ww2nv\"" Jul 12 15:24:16.968607 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.968543 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:24:16.968659 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.968602 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 12 15:24:16.968972 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.968956 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 12 15:24:16.969024 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.968989 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 12 15:24:16.969112 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.969096 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 12 15:24:16.970558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.970539 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zsshq\"" Jul 12 15:24:16.970653 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.970600 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 12 15:24:16.980005 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.979982 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 12 15:24:16.985515 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.985492 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-lk5sx"] Jul 12 15:24:16.997698 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:16.997671 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7"] Jul 12 15:24:17.015328 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015302 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.015484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.015484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015388 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mzvr5\" (UniqueName: \"kubernetes.io/projected/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-kube-api-access-mzvr5\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.015484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015416 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-config\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.015484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015441 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gftgh\" (UniqueName: \"kubernetes.io/projected/88c42517-0b00-4aae-8dde-6805b3dcb6b2-kube-api-access-gftgh\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.015484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015467 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88c42517-0b00-4aae-8dde-6805b3dcb6b2-serving-cert\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.015667 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.015507 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-trusted-ca\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.031843 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.031816 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb"] Jul 12 15:24:17.033666 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.033650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" Jul 12 15:24:17.036312 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.036293 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 12 15:24:17.036312 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.036303 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv"] Jul 12 15:24:17.036497 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.036481 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-qp7wx\"" Jul 12 15:24:17.036730 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.036717 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:24:17.038230 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.038215 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.041043 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.041019 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 12 15:24:17.041183 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.041099 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:24:17.041183 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.041112 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 12 15:24:17.041183 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.041115 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nf6n2\"" Jul 12 15:24:17.045273 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.045250 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb"] Jul 12 15:24:17.064484 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.064458 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv"] Jul 12 15:24:17.116625 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116585 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.116825 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116630 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mzvr5\" (UniqueName: \"kubernetes.io/projected/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-kube-api-access-mzvr5\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.116825 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116759 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-config\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.116825 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gftgh\" (UniqueName: \"kubernetes.io/projected/88c42517-0b00-4aae-8dde-6805b3dcb6b2-kube-api-access-gftgh\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.116982 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116828 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g2zdf\" (UniqueName: \"kubernetes.io/projected/8d57e8df-344b-4de6-b59b-df18b8de64f5-kube-api-access-g2zdf\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.116982 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116859 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88c42517-0b00-4aae-8dde-6805b3dcb6b2-serving-cert\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.116982 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116882 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-trusted-ca\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.116982 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.116910 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.117192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.117019 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.117192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.117094 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-df4qt\" (UniqueName: \"kubernetes.io/projected/65c4e4d8-19d5-497c-96de-abfb7986b36a-kube-api-access-df4qt\") pod \"volume-data-source-validator-78495597f6-c88vb\" (UID: \"65c4e4d8-19d5-497c-96de-abfb7986b36a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" Jul 12 15:24:17.117192 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.117180 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:17.117298 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.117260 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:17.617240588 +0000 UTC m=+235.326048160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:17.117536 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.117511 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.117536 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.117530 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-config\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.117815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.117799 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/88c42517-0b00-4aae-8dde-6805b3dcb6b2-trusted-ca\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.119425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.119398 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/88c42517-0b00-4aae-8dde-6805b3dcb6b2-serving-cert\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.126711 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.126689 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mzvr5\" (UniqueName: \"kubernetes.io/projected/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-kube-api-access-mzvr5\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.126821 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.126734 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gftgh\" (UniqueName: \"kubernetes.io/projected/88c42517-0b00-4aae-8dde-6805b3dcb6b2-kube-api-access-gftgh\") pod \"console-operator-7866497494-lk5sx\" (UID: \"88c42517-0b00-4aae-8dde-6805b3dcb6b2\") " pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.218248 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.218170 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g2zdf\" (UniqueName: \"kubernetes.io/projected/8d57e8df-344b-4de6-b59b-df18b8de64f5-kube-api-access-g2zdf\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.218248 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.218206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.218248 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.218246 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-df4qt\" (UniqueName: \"kubernetes.io/projected/65c4e4d8-19d5-497c-96de-abfb7986b36a-kube-api-access-df4qt\") pod \"volume-data-source-validator-78495597f6-c88vb\" (UID: \"65c4e4d8-19d5-497c-96de-abfb7986b36a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" Jul 12 15:24:17.218446 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.218346 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 15:24:17.218446 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.218426 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls podName:8d57e8df-344b-4de6-b59b-df18b8de64f5 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:17.718405556 +0000 UTC m=+235.427213129 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls") pod "cluster-samples-operator-65db858894-nphvv" (UID: "8d57e8df-344b-4de6-b59b-df18b8de64f5") : secret "samples-operator-tls" not found Jul 12 15:24:17.229288 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.229256 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-df4qt\" (UniqueName: \"kubernetes.io/projected/65c4e4d8-19d5-497c-96de-abfb7986b36a-kube-api-access-df4qt\") pod \"volume-data-source-validator-78495597f6-c88vb\" (UID: \"65c4e4d8-19d5-497c-96de-abfb7986b36a\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" Jul 12 15:24:17.229288 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.229281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g2zdf\" (UniqueName: \"kubernetes.io/projected/8d57e8df-344b-4de6-b59b-df18b8de64f5-kube-api-access-g2zdf\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.274171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.274129 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:17.344171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.344130 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" Jul 12 15:24:17.401328 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.401294 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-lk5sx"] Jul 12 15:24:17.406831 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:17.406806 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c42517_0b00_4aae_8dde_6805b3dcb6b2.slice/crio-c8da534808ce1e60b7a673409136d557ef20219e30f1fcec6405e59bd0c5dbe4 WatchSource:0}: Error finding container c8da534808ce1e60b7a673409136d557ef20219e30f1fcec6405e59bd0c5dbe4: Status 404 returned error can't find the container with id c8da534808ce1e60b7a673409136d557ef20219e30f1fcec6405e59bd0c5dbe4 Jul 12 15:24:17.472624 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.472543 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb"] Jul 12 15:24:17.475349 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:17.475327 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c4e4d8_19d5_497c_96de_abfb7986b36a.slice/crio-b4eeea372d118b92d3a6cff36491548e391bee5068d86b0cd8836234cd8247ac WatchSource:0}: Error finding container b4eeea372d118b92d3a6cff36491548e391bee5068d86b0cd8836234cd8247ac: Status 404 returned error can't find the container with id b4eeea372d118b92d3a6cff36491548e391bee5068d86b0cd8836234cd8247ac Jul 12 15:24:17.573519 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.573484 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lk5sx" event={"ID":"88c42517-0b00-4aae-8dde-6805b3dcb6b2","Type":"ContainerStarted","Data":"c8da534808ce1e60b7a673409136d557ef20219e30f1fcec6405e59bd0c5dbe4"} Jul 12 15:24:17.574491 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.574470 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" event={"ID":"65c4e4d8-19d5-497c-96de-abfb7986b36a","Type":"ContainerStarted","Data":"b4eeea372d118b92d3a6cff36491548e391bee5068d86b0cd8836234cd8247ac"} Jul 12 15:24:17.621366 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.621328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:17.621530 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.621479 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:17.621569 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.621540 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:18.621524054 +0000 UTC m=+236.330331627 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:17.722399 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:17.722365 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:17.722510 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.722498 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 15:24:17.722569 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:17.722559 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls podName:8d57e8df-344b-4de6-b59b-df18b8de64f5 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:18.722544693 +0000 UTC m=+236.431352265 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls") pod "cluster-samples-operator-65db858894-nphvv" (UID: "8d57e8df-344b-4de6-b59b-df18b8de64f5") : secret "samples-operator-tls" not found Jul 12 15:24:18.631115 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:18.631076 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:18.631583 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:18.631242 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:18.631583 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:18.631330 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:20.631308305 +0000 UTC m=+238.340115893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:18.731863 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:18.731825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:18.732015 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:18.731977 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 15:24:18.732065 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:18.732060 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls podName:8d57e8df-344b-4de6-b59b-df18b8de64f5 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:20.73202925 +0000 UTC m=+238.440836823 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls") pod "cluster-samples-operator-65db858894-nphvv" (UID: "8d57e8df-344b-4de6-b59b-df18b8de64f5") : secret "samples-operator-tls" not found Jul 12 15:24:19.583593 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.583557 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lk5sx" event={"ID":"88c42517-0b00-4aae-8dde-6805b3dcb6b2","Type":"ContainerStarted","Data":"d750bbe96b1dd9b86e8431358442e84174ac55684796c773e0069f38645c484a"} Jul 12 15:24:19.583775 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.583759 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:19.585284 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.585256 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" event={"ID":"65c4e4d8-19d5-497c-96de-abfb7986b36a","Type":"ContainerStarted","Data":"ea9aeac9e030df340783762b40c464e7c9c462e0763e2e8a13baa68c76c2e404"} Jul 12 15:24:19.585411 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.585273 2567 patch_prober.go:28] interesting pod/console-operator-7866497494-lk5sx container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.134.0.8:8443/readyz\": dial tcp 10.134.0.8:8443: connect: connection refused" start-of-body= Jul 12 15:24:19.585411 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.585324 2567 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podUID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" containerName="console-operator" probeResult="failure" output="Get \"https://10.134.0.8:8443/readyz\": dial tcp 10.134.0.8:8443: connect: connection refused" Jul 12 15:24:19.599855 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.599808 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podStartSLOduration=1.5194175269999999 podStartE2EDuration="3.599792943s" totalImagesPullingTime="2.080375416s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:19.599543563 +0000 UTC m=+237.308351157" watchObservedRunningTime="2026-07-12 15:24:19.599792943 +0000 UTC m=+237.308600583" Jul 12 15:24:19.614110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:19.614072 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-c88vb" podStartSLOduration=1.347771617 podStartE2EDuration="2.614039277s" totalImagesPullingTime="1.26626766s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:19.613457369 +0000 UTC m=+237.322264982" watchObservedRunningTime="2026-07-12 15:24:19.614039277 +0000 UTC m=+237.322846871" Jul 12 15:24:20.592091 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.592062 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/0.log" Jul 12 15:24:20.592553 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.592100 2567 generic.go:356] "Generic (PLEG): container finished" podID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" containerID="d750bbe96b1dd9b86e8431358442e84174ac55684796c773e0069f38645c484a" exitCode=255 Jul 12 15:24:20.592553 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.592131 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lk5sx" event={"ID":"88c42517-0b00-4aae-8dde-6805b3dcb6b2","Type":"ContainerDied","Data":"d750bbe96b1dd9b86e8431358442e84174ac55684796c773e0069f38645c484a"} Jul 12 15:24:20.592553 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.592430 2567 scope.go:117] "RemoveContainer" containerID="d750bbe96b1dd9b86e8431358442e84174ac55684796c773e0069f38645c484a" Jul 12 15:24:20.648889 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.648862 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:20.649000 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:20.648982 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:20.649107 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:20.649039 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:24.649026171 +0000 UTC m=+242.357833742 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:20.749653 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:20.749613 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:20.749846 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:20.749800 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 15:24:20.749915 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:20.749891 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls podName:8d57e8df-344b-4de6-b59b-df18b8de64f5 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:24.749868344 +0000 UTC m=+242.458675921 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls") pod "cluster-samples-operator-65db858894-nphvv" (UID: "8d57e8df-344b-4de6-b59b-df18b8de64f5") : secret "samples-operator-tls" not found Jul 12 15:24:21.596685 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.596661 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:24:21.597039 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.597023 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/0.log" Jul 12 15:24:21.597114 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.597070 2567 generic.go:356] "Generic (PLEG): container finished" podID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" exitCode=255 Jul 12 15:24:21.597148 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.597125 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lk5sx" event={"ID":"88c42517-0b00-4aae-8dde-6805b3dcb6b2","Type":"ContainerDied","Data":"6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87"} Jul 12 15:24:21.597184 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.597178 2567 scope.go:117] "RemoveContainer" containerID="d750bbe96b1dd9b86e8431358442e84174ac55684796c773e0069f38645c484a" Jul 12 15:24:21.597411 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.597388 2567 scope.go:117] "RemoveContainer" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" Jul 12 15:24:21.597578 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:21.597559 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lk5sx_openshift-console-operator(88c42517-0b00-4aae-8dde-6805b3dcb6b2)\"" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podUID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" Jul 12 15:24:21.883225 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.883127 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-m2tkh"] Jul 12 15:24:21.886066 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.886030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:21.888739 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.888719 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 12 15:24:21.888830 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.888717 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 12 15:24:21.888830 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.888753 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 12 15:24:21.889109 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.889091 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 12 15:24:21.889990 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.889970 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4k8zs\"" Jul 12 15:24:21.896323 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.896300 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2tkh"] Jul 12 15:24:21.959549 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.959518 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/406610d9-4c13-4368-9059-776813671305-crio-socket\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:21.959703 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.959563 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:21.959703 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.959596 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:21.959703 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.959623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w8lk6\" (UniqueName: \"kubernetes.io/projected/406610d9-4c13-4368-9059-776813671305-kube-api-access-w8lk6\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:21.959703 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:21.959642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/406610d9-4c13-4368-9059-776813671305-data-volume\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060297 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060264 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/406610d9-4c13-4368-9059-776813671305-crio-socket\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060320 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060350 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060377 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w8lk6\" (UniqueName: \"kubernetes.io/projected/406610d9-4c13-4368-9059-776813671305-kube-api-access-w8lk6\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060390 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/406610d9-4c13-4368-9059-776813671305-crio-socket\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060404 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/406610d9-4c13-4368-9059-776813671305-data-volume\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.060528 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:22.060542 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.060542 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.060826 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.060600 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:22.560582257 +0000 UTC m=+240.269389833 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : configmap "kube-rbac-proxy" not found Jul 12 15:24:22.060826 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.060620 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:22.560611005 +0000 UTC m=+240.269418579 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.060826 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.060798 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/406610d9-4c13-4368-9059-776813671305-data-volume\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.076999 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.076972 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w8lk6\" (UniqueName: \"kubernetes.io/projected/406610d9-4c13-4368-9059-776813671305-kube-api-access-w8lk6\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.129498 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.129464 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6"] Jul 12 15:24:22.131567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.131552 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" Jul 12 15:24:22.134266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.134209 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 12 15:24:22.134266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.134215 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 12 15:24:22.134425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.134381 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-7j4rz\"" Jul 12 15:24:22.140311 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.140293 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6"] Jul 12 15:24:22.261971 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.261938 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4x8fx\" (UniqueName: \"kubernetes.io/projected/d6f1420e-38bd-48e2-9c5d-7d34435f7f33-kube-api-access-4x8fx\") pod \"migrator-7f59b54f6c-4hqf6\" (UID: \"d6f1420e-38bd-48e2-9c5d-7d34435f7f33\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" Jul 12 15:24:22.363355 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.363306 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4x8fx\" (UniqueName: \"kubernetes.io/projected/d6f1420e-38bd-48e2-9c5d-7d34435f7f33-kube-api-access-4x8fx\") pod \"migrator-7f59b54f6c-4hqf6\" (UID: \"d6f1420e-38bd-48e2-9c5d-7d34435f7f33\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" Jul 12 15:24:22.374335 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.374309 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4x8fx\" (UniqueName: \"kubernetes.io/projected/d6f1420e-38bd-48e2-9c5d-7d34435f7f33-kube-api-access-4x8fx\") pod \"migrator-7f59b54f6c-4hqf6\" (UID: \"d6f1420e-38bd-48e2-9c5d-7d34435f7f33\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" Jul 12 15:24:22.440128 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.440030 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" Jul 12 15:24:22.560951 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.560917 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6"] Jul 12 15:24:22.564313 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:22.564281 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f1420e_38bd_48e2_9c5d_7d34435f7f33.slice/crio-d010addd15e64867d260431833280a69934454a78d3b0fbb8d4757c72beebbab WatchSource:0}: Error finding container d010addd15e64867d260431833280a69934454a78d3b0fbb8d4757c72beebbab: Status 404 returned error can't find the container with id d010addd15e64867d260431833280a69934454a78d3b0fbb8d4757c72beebbab Jul 12 15:24:22.565581 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.565561 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.565657 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.565594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:22.565718 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.565702 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:22.565769 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.565726 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.565819 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.565767 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:23.565746097 +0000 UTC m=+241.274553719 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : configmap "kube-rbac-proxy" not found Jul 12 15:24:22.565819 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.565785 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:23.565777157 +0000 UTC m=+241.274584735 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:22.600979 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.600956 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:24:22.601378 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.601362 2567 scope.go:117] "RemoveContainer" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" Jul 12 15:24:22.601535 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:22.601517 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lk5sx_openshift-console-operator(88c42517-0b00-4aae-8dde-6805b3dcb6b2)\"" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podUID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" Jul 12 15:24:22.602237 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:22.602212 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" event={"ID":"d6f1420e-38bd-48e2-9c5d-7d34435f7f33","Type":"ContainerStarted","Data":"d010addd15e64867d260431833280a69934454a78d3b0fbb8d4757c72beebbab"} Jul 12 15:24:23.572874 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:23.572833 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:23.573083 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:23.572889 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:23.573083 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:23.572964 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:23.573083 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:23.573027 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:23.573083 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:23.573040 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:25.573021083 +0000 UTC m=+243.281828659 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : configmap "kube-rbac-proxy" not found Jul 12 15:24:23.573265 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:23.573097 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:25.573081679 +0000 UTC m=+243.281889252 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:23.648154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:23.648119 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5xx64_65cba0d8-d7a9-4d40-893b-b082b5f72ae4/dns-node-resolver/0.log" Jul 12 15:24:24.288764 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.288687 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mfc6c"] Jul 12 15:24:24.290703 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.290687 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.293476 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.293455 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 12 15:24:24.294720 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.294703 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 12 15:24:24.294862 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.294715 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 12 15:24:24.294862 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.294717 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-4qhdj\"" Jul 12 15:24:24.294862 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.294740 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 12 15:24:24.299723 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.299705 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mfc6c"] Jul 12 15:24:24.378831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.378802 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-key\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.379003 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.378838 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.379003 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.378864 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvk9k\" (UniqueName: \"kubernetes.io/projected/3ea19724-cd1c-4334-8b8c-aa0655b667d0-kube-api-access-cvk9k\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.480266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.480216 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-key\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.480266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.480262 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.480266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.480280 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvk9k\" (UniqueName: \"kubernetes.io/projected/3ea19724-cd1c-4334-8b8c-aa0655b667d0-kube-api-access-cvk9k\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.480852 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.480825 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-cabundle\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.482750 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.482726 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/3ea19724-cd1c-4334-8b8c-aa0655b667d0-signing-key\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.488517 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.488489 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvk9k\" (UniqueName: \"kubernetes.io/projected/3ea19724-cd1c-4334-8b8c-aa0655b667d0-kube-api-access-cvk9k\") pod \"service-ca-79f68b9f4f-mfc6c\" (UID: \"3ea19724-cd1c-4334-8b8c-aa0655b667d0\") " pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.599194 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.599170 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" Jul 12 15:24:24.610154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.610120 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" event={"ID":"d6f1420e-38bd-48e2-9c5d-7d34435f7f33","Type":"ContainerStarted","Data":"d3b9354ff7ea588ec4c83d66d9b80dc0caec19532fb1111b042f429e3a43a33f"} Jul 12 15:24:24.610265 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.610186 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" event={"ID":"d6f1420e-38bd-48e2-9c5d-7d34435f7f33","Type":"ContainerStarted","Data":"d7931eb95f241004dc6eb83e3d6cc8cc0e3a55c5acf7b638e77d58d0dde57cb4"} Jul 12 15:24:24.627447 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.627394 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-4hqf6" podStartSLOduration=1.445608883 podStartE2EDuration="2.627377949s" totalImagesPullingTime="1.181769066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:24.627031872 +0000 UTC m=+242.335839466" watchObservedRunningTime="2026-07-12 15:24:24.627377949 +0000 UTC m=+242.336185594" Jul 12 15:24:24.648373 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.648290 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7g55n_ec638ef4-7065-4b9a-9ac7-102a3ad10865/node-ca/0.log" Jul 12 15:24:24.682531 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.682493 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:24.682661 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:24.682642 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:24.682726 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:24.682716 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:32.682700758 +0000 UTC m=+250.391508333 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:24.719003 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.718969 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-79f68b9f4f-mfc6c"] Jul 12 15:24:24.722011 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:24.721977 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ea19724_cd1c_4334_8b8c_aa0655b667d0.slice/crio-3147beaba786497c553633af22916b244d4c06a7591674ac4ff2d2e9b27ddca7 WatchSource:0}: Error finding container 3147beaba786497c553633af22916b244d4c06a7591674ac4ff2d2e9b27ddca7: Status 404 returned error can't find the container with id 3147beaba786497c553633af22916b244d4c06a7591674ac4ff2d2e9b27ddca7 Jul 12 15:24:24.783603 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:24.783578 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:24.783738 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:24.783723 2567 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 12 15:24:24.783785 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:24.783777 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls podName:8d57e8df-344b-4de6-b59b-df18b8de64f5 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:32.783762933 +0000 UTC m=+250.492570504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls") pod "cluster-samples-operator-65db858894-nphvv" (UID: "8d57e8df-344b-4de6-b59b-df18b8de64f5") : secret "samples-operator-tls" not found Jul 12 15:24:25.449420 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:25.449388 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5ds8_1d63f4ab-9cfd-46d1-9dc5-0da56873da41/serve-healthcheck-canary/0.log" Jul 12 15:24:25.590202 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:25.590163 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:25.590202 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:25.590206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:25.590444 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:25.590309 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:25.590444 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:25.590320 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:25.590444 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:25.590385 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:29.590365675 +0000 UTC m=+247.299173263 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : configmap "kube-rbac-proxy" not found Jul 12 15:24:25.590444 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:25.590404 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:29.590394699 +0000 UTC m=+247.299202272 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:25.615569 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:25.615519 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" event={"ID":"3ea19724-cd1c-4334-8b8c-aa0655b667d0","Type":"ContainerStarted","Data":"3147beaba786497c553633af22916b244d4c06a7591674ac4ff2d2e9b27ddca7"} Jul 12 15:24:26.620314 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:26.620277 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" event={"ID":"3ea19724-cd1c-4334-8b8c-aa0655b667d0","Type":"ContainerStarted","Data":"bb2e5b5ea43d53fa61bf295653e7b50cb2f4bb68472a99804b06d120e9ba2f0f"} Jul 12 15:24:26.638172 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:26.638114 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-79f68b9f4f-mfc6c" podStartSLOduration=0.86458879 podStartE2EDuration="2.638094011s" totalImagesPullingTime="1.773505221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:26.637083931 +0000 UTC m=+244.345891525" watchObservedRunningTime="2026-07-12 15:24:26.638094011 +0000 UTC m=+244.346901608" Jul 12 15:24:27.274592 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:27.274492 2567 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:27.274868 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:27.274855 2567 scope.go:117] "RemoveContainer" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" Jul 12 15:24:27.275012 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:27.274994 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lk5sx_openshift-console-operator(88c42517-0b00-4aae-8dde-6805b3dcb6b2)\"" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podUID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" Jul 12 15:24:29.583942 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:29.583902 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:29.584363 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:29.584280 2567 scope.go:117] "RemoveContainer" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" Jul 12 15:24:29.584440 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:29.584422 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-lk5sx_openshift-console-operator(88c42517-0b00-4aae-8dde-6805b3dcb6b2)\"" pod="openshift-console-operator/console-operator-7866497494-lk5sx" podUID="88c42517-0b00-4aae-8dde-6805b3dcb6b2" Jul 12 15:24:29.626286 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:29.626256 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:29.626414 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:29.626294 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:29.626414 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:29.626374 2567 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 12 15:24:29.626483 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:29.626432 2567 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 12 15:24:29.626483 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:29.626454 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:37.62643693 +0000 UTC m=+255.335244502 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : configmap "kube-rbac-proxy" not found Jul 12 15:24:29.626483 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:29.626473 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls podName:406610d9-4c13-4368-9059-776813671305 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:37.626462196 +0000 UTC m=+255.335269768 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls") pod "insights-runtime-extractor-m2tkh" (UID: "406610d9-4c13-4368-9059-776813671305") : secret "insights-runtime-extractor-tls" not found Jul 12 15:24:32.754127 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:32.754084 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:32.754614 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:32.754266 2567 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:32.754614 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:32.754348 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls podName:44f318e0-6b39-4c0d-b348-afe3f1e3c05b nodeName:}" failed. No retries permitted until 2026-07-12 15:24:48.75432694 +0000 UTC m=+266.463134535 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-jdjc7" (UID: "44f318e0-6b39-4c0d-b348-afe3f1e3c05b") : secret "cluster-monitoring-operator-tls" not found Jul 12 15:24:32.854985 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:32.854951 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:32.857636 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:32.857604 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/8d57e8df-344b-4de6-b59b-df18b8de64f5-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-nphvv\" (UID: \"8d57e8df-344b-4de6-b59b-df18b8de64f5\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:32.951914 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:32.951887 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-nf6n2\"" Jul 12 15:24:32.960244 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:32.960223 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" Jul 12 15:24:33.087736 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:33.087700 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv"] Jul 12 15:24:33.643388 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:33.643349 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" event={"ID":"8d57e8df-344b-4de6-b59b-df18b8de64f5","Type":"ContainerStarted","Data":"e4f2bb69f1e59ba5617ad5531de415bd1aa6fc5d0d772e8f2f9312f49029d38c"} Jul 12 15:24:34.669344 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:34.669313 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:24:34.669837 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:34.669498 2567 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 12 15:24:34.669837 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:34.669573 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs podName:c18abc6a-0f18-4948-8c24-6bda26a89278 nodeName:}" failed. No retries permitted until 2026-07-12 15:26:36.669554033 +0000 UTC m=+374.378361632 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs") pod "network-metrics-daemon-5x2w7" (UID: "c18abc6a-0f18-4948-8c24-6bda26a89278") : secret "metrics-daemon-secret" not found Jul 12 15:24:35.652002 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:35.651966 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" event={"ID":"8d57e8df-344b-4de6-b59b-df18b8de64f5","Type":"ContainerStarted","Data":"bd5168ff88fd586b3f3e022560932508843d202b44be589b025f58f916e6b41a"} Jul 12 15:24:35.652425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:35.652029 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" event={"ID":"8d57e8df-344b-4de6-b59b-df18b8de64f5","Type":"ContainerStarted","Data":"78b805e31728e62c0e7c561af94456fc4e4758d43ad7d1ce69a139f06171df0c"} Jul 12 15:24:35.670102 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:35.670034 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-nphvv" podStartSLOduration=16.986263172 podStartE2EDuration="18.670019631s" totalImagesPullingTime="1.683756459s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:35.668909246 +0000 UTC m=+253.377716842" watchObservedRunningTime="2026-07-12 15:24:35.670019631 +0000 UTC m=+253.378827224" Jul 12 15:24:37.692429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.692393 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:37.692429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.692439 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:37.692939 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.692904 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/406610d9-4c13-4368-9059-776813671305-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:37.694844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.694818 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/406610d9-4c13-4368-9059-776813671305-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-m2tkh\" (UID: \"406610d9-4c13-4368-9059-776813671305\") " pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:37.797574 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.797534 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-4k8zs\"" Jul 12 15:24:37.805640 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.805613 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-m2tkh" Jul 12 15:24:37.934350 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:37.934320 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-m2tkh"] Jul 12 15:24:37.937687 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:37.937659 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod406610d9_4c13_4368_9059_776813671305.slice/crio-affad9167d6614002cdec3e0f89969e31b1c10d052a9c9571201adf9724bfb3f WatchSource:0}: Error finding container affad9167d6614002cdec3e0f89969e31b1c10d052a9c9571201adf9724bfb3f: Status 404 returned error can't find the container with id affad9167d6614002cdec3e0f89969e31b1c10d052a9c9571201adf9724bfb3f Jul 12 15:24:38.665638 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:38.665600 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2tkh" event={"ID":"406610d9-4c13-4368-9059-776813671305","Type":"ContainerStarted","Data":"affad9167d6614002cdec3e0f89969e31b1c10d052a9c9571201adf9724bfb3f"} Jul 12 15:24:41.675760 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:41.675727 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2tkh" event={"ID":"406610d9-4c13-4368-9059-776813671305","Type":"ContainerStarted","Data":"9647c9bf9ea9a43a4110fe1fb00686c71692ccb72c826182749ea579c0d80023"} Jul 12 15:24:41.840429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:41.840400 2567 scope.go:117] "RemoveContainer" containerID="6de8864484e8bf4ca9295839b198d5e7184a8ad52ad482294f9766d1bad2bc87" Jul 12 15:24:42.682979 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:42.682889 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2tkh" event={"ID":"406610d9-4c13-4368-9059-776813671305","Type":"ContainerStarted","Data":"f7adbd9b40c90df94662d7e09175c3214365b11915646b090c873e5cc3f9ff32"} Jul 12 15:24:42.684825 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:42.684803 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:24:42.684951 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:42.684894 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-lk5sx" event={"ID":"88c42517-0b00-4aae-8dde-6805b3dcb6b2","Type":"ContainerStarted","Data":"e92b2037ff6311520d35d9be8d14271a3e10da4b9345df9892555f54040109e0"} Jul 12 15:24:42.685244 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:42.685220 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:42.690470 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:42.690448 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-lk5sx" Jul 12 15:24:45.100394 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.100364 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-fmjc5"] Jul 12 15:24:45.103399 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.103382 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:24:45.106236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.106035 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 12 15:24:45.106524 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.106506 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-spvnb\"" Jul 12 15:24:45.106810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.106798 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 12 15:24:45.112359 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.112335 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fmjc5"] Jul 12 15:24:45.147942 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.147907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hjzq\" (UniqueName: \"kubernetes.io/projected/314047f5-49d6-4509-ab39-b49f0e4e6ca2-kube-api-access-7hjzq\") pod \"downloads-68c9cbb497-fmjc5\" (UID: \"314047f5-49d6-4509-ab39-b49f0e4e6ca2\") " pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:24:45.248718 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.248678 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7hjzq\" (UniqueName: \"kubernetes.io/projected/314047f5-49d6-4509-ab39-b49f0e4e6ca2-kube-api-access-7hjzq\") pod \"downloads-68c9cbb497-fmjc5\" (UID: \"314047f5-49d6-4509-ab39-b49f0e4e6ca2\") " pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:24:45.259769 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.259739 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hjzq\" (UniqueName: \"kubernetes.io/projected/314047f5-49d6-4509-ab39-b49f0e4e6ca2-kube-api-access-7hjzq\") pod \"downloads-68c9cbb497-fmjc5\" (UID: \"314047f5-49d6-4509-ab39-b49f0e4e6ca2\") " pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:24:45.412633 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.412543 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:24:45.541124 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.541090 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-fmjc5"] Jul 12 15:24:45.545369 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:45.545344 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod314047f5_49d6_4509_ab39_b49f0e4e6ca2.slice/crio-0068603fd9411f8b190c2a239eb9f96a088a0321f1e595dfb2724addb72957ec WatchSource:0}: Error finding container 0068603fd9411f8b190c2a239eb9f96a088a0321f1e595dfb2724addb72957ec: Status 404 returned error can't find the container with id 0068603fd9411f8b190c2a239eb9f96a088a0321f1e595dfb2724addb72957ec Jul 12 15:24:45.696141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.696037 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fmjc5" event={"ID":"314047f5-49d6-4509-ab39-b49f0e4e6ca2","Type":"ContainerStarted","Data":"0068603fd9411f8b190c2a239eb9f96a088a0321f1e595dfb2724addb72957ec"} Jul 12 15:24:45.697881 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.697859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-m2tkh" event={"ID":"406610d9-4c13-4368-9059-776813671305","Type":"ContainerStarted","Data":"525512b4c1f76880f951a5668e566a5931bc9aaad1365f72fb2f9c7496e66978"} Jul 12 15:24:45.714369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:45.714316 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-m2tkh" podStartSLOduration=18.093429705 podStartE2EDuration="24.714298297s" totalImagesPullingTime="6.620868592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 15:24:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:45.713822884 +0000 UTC m=+263.422630481" watchObservedRunningTime="2026-07-12 15:24:45.714298297 +0000 UTC m=+263.423105892" Jul 12 15:24:48.776114 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:48.776040 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:48.778517 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:48.778492 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/44f318e0-6b39-4c0d-b348-afe3f1e3c05b-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-jdjc7\" (UID: \"44f318e0-6b39-4c0d-b348-afe3f1e3c05b\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:48.782816 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:48.782790 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-zsshq\"" Jul 12 15:24:48.790492 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:48.790474 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" Jul 12 15:24:48.933118 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:48.933079 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7"] Jul 12 15:24:48.936318 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:48.936281 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44f318e0_6b39_4c0d_b348_afe3f1e3c05b.slice/crio-d8320481474a11a93ef8939df44ca80d01b77dc15caf5112a1e1b07a7d6b5b3e WatchSource:0}: Error finding container d8320481474a11a93ef8939df44ca80d01b77dc15caf5112a1e1b07a7d6b5b3e: Status 404 returned error can't find the container with id d8320481474a11a93ef8939df44ca80d01b77dc15caf5112a1e1b07a7d6b5b3e Jul 12 15:24:49.714525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:49.714489 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" event={"ID":"44f318e0-6b39-4c0d-b348-afe3f1e3c05b","Type":"ContainerStarted","Data":"d8320481474a11a93ef8939df44ca80d01b77dc15caf5112a1e1b07a7d6b5b3e"} Jul 12 15:24:51.432958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.432921 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r"] Jul 12 15:24:51.436668 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.436646 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:51.440097 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.440072 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 12 15:24:51.440226 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.440095 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-hgj89\"" Jul 12 15:24:51.455758 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.455727 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r"] Jul 12 15:24:51.499066 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.499013 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7rd6r\" (UID: \"d78631af-aba3-42c3-aa6c-0c3721d050e8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:51.599714 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.599675 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7rd6r\" (UID: \"d78631af-aba3-42c3-aa6c-0c3721d050e8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:51.599889 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:51.599813 2567 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 12 15:24:51.599961 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:51.599888 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates podName:d78631af-aba3-42c3-aa6c-0c3721d050e8 nodeName:}" failed. No retries permitted until 2026-07-12 15:24:52.09986482 +0000 UTC m=+269.808672417 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-7rd6r" (UID: "d78631af-aba3-42c3-aa6c-0c3721d050e8") : secret "prometheus-operator-admission-webhook-tls" not found Jul 12 15:24:51.724949 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.724859 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" event={"ID":"44f318e0-6b39-4c0d-b348-afe3f1e3c05b","Type":"ContainerStarted","Data":"02d947597a772f8495a295ef7a7f889a1f75e7f5ababa03572a4cb5897fd580e"} Jul 12 15:24:51.740709 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:51.740649 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-jdjc7" podStartSLOduration=33.760887884 podStartE2EDuration="35.740626905s" totalImagesPullingTime="1.979739021s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:51.739350754 +0000 UTC m=+269.448158348" watchObservedRunningTime="2026-07-12 15:24:51.740626905 +0000 UTC m=+269.449434502" Jul 12 15:24:52.102749 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:52.102709 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7rd6r\" (UID: \"d78631af-aba3-42c3-aa6c-0c3721d050e8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:52.105654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:52.105629 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d78631af-aba3-42c3-aa6c-0c3721d050e8-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-7rd6r\" (UID: \"d78631af-aba3-42c3-aa6c-0c3721d050e8\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:52.349366 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:52.349325 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:52.501466 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:52.501436 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r"] Jul 12 15:24:52.504395 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:52.504365 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd78631af_aba3_42c3_aa6c_0c3721d050e8.slice/crio-fff250d2a101c04139e88db6fa411fd982f59a5f58728169dad7a58b31fb4fa2 WatchSource:0}: Error finding container fff250d2a101c04139e88db6fa411fd982f59a5f58728169dad7a58b31fb4fa2: Status 404 returned error can't find the container with id fff250d2a101c04139e88db6fa411fd982f59a5f58728169dad7a58b31fb4fa2 Jul 12 15:24:52.730406 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:52.730323 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" event={"ID":"d78631af-aba3-42c3-aa6c-0c3721d050e8","Type":"ContainerStarted","Data":"fff250d2a101c04139e88db6fa411fd982f59a5f58728169dad7a58b31fb4fa2"} Jul 12 15:24:54.739478 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:54.739427 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" event={"ID":"d78631af-aba3-42c3-aa6c-0c3721d050e8","Type":"ContainerStarted","Data":"0d7592eef8d3dab8db0ca1af0e253c467e3b05ae20db422f48cda3f76215bf7f"} Jul 12 15:24:54.739946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:54.739671 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:54.744772 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:54.744747 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" Jul 12 15:24:54.755157 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:54.755114 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-7rd6r" podStartSLOduration=2.48172159 podStartE2EDuration="3.755102326s" totalImagesPullingTime="1.273380736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:24:54.754253369 +0000 UTC m=+272.463060965" watchObservedRunningTime="2026-07-12 15:24:54.755102326 +0000 UTC m=+272.463909919" Jul 12 15:24:55.109476 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.109430 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:24:55.165906 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.165869 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:24:55.166100 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.166004 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.169228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.169199 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 12 15:24:55.169387 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.169200 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 12 15:24:55.170429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.170224 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 12 15:24:55.170429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.170231 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 12 15:24:55.170429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.170289 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-dt6sj\"" Jul 12 15:24:55.170429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.170342 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 12 15:24:55.229257 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229225 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.229257 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229262 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nx59v\" (UniqueName: \"kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.229510 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229289 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.229510 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229316 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.229510 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229343 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.229510 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.229434 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330418 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330384 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330604 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330441 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330604 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330470 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nx59v\" (UniqueName: \"kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330604 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330604 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330506 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.330604 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.330521 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.331991 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.331966 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.332146 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.331968 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.332146 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.332037 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.333392 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.333368 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.333713 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.333680 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.340021 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.339998 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nx59v\" (UniqueName: \"kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v\") pod \"console-679b585cd6-7fw8c\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.476729 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.476537 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:24:55.653966 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.653933 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:24:55.657943 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:24:55.657909 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51148455_5098_492c_9840_fdf52220f5ad.slice/crio-d3caaf5d98e68f4140d304df4b05c2059943a910fdbc5849b45985c18eac6e01 WatchSource:0}: Error finding container d3caaf5d98e68f4140d304df4b05c2059943a910fdbc5849b45985c18eac6e01: Status 404 returned error can't find the container with id d3caaf5d98e68f4140d304df4b05c2059943a910fdbc5849b45985c18eac6e01 Jul 12 15:24:55.745173 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:55.745088 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b585cd6-7fw8c" event={"ID":"51148455-5098-492c-9840-fdf52220f5ad","Type":"ContainerStarted","Data":"d3caaf5d98e68f4140d304df4b05c2059943a910fdbc5849b45985c18eac6e01"} Jul 12 15:24:58.298694 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.298659 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:24:58.329339 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.329302 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:24:58.329501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.329413 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.338678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.338636 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 12 15:24:58.355256 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355234 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355264 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-84vk6\" (UniqueName: \"kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355380 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355551 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355407 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355551 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.355551 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.355463 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456199 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456243 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456311 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456413 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456456 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-84vk6\" (UniqueName: \"kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.456719 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.456600 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.457108 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.457032 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.457224 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.457116 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.457224 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.457207 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.457320 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.457239 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.459263 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.459218 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.459431 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.459409 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.464483 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.464458 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-84vk6\" (UniqueName: \"kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6\") pod \"console-bccbdd6fc-7ql29\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:58.640881 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:58.640841 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:24:59.821372 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.821341 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk"] Jul 12 15:24:59.843879 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.843148 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ff8fp"] Jul 12 15:24:59.843879 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.843456 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.848508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.847172 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 12 15:24:59.848508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.847451 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4v84d\"" Jul 12 15:24:59.848508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.847679 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 12 15:24:59.848508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.847876 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 12 15:24:59.866007 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.865975 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk"] Jul 12 15:24:59.866155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.866036 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-7j5xx"] Jul 12 15:24:59.866155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.866122 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.869113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869072 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 12 15:24:59.869245 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869225 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 12 15:24:59.869362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869345 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 12 15:24:59.869449 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869430 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.869508 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869486 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d14f178-0fd1-4143-a142-68b649c2b61b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.869565 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869524 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-nc9s6\"" Jul 12 15:24:59.869565 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869551 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdfpz\" (UniqueName: \"kubernetes.io/projected/5d14f178-0fd1-4143-a142-68b649c2b61b-kube-api-access-tdfpz\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.869666 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.869637 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.885964 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.885942 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ff8fp"] Jul 12 15:24:59.886119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.886082 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.888707 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.888560 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 12 15:24:59.888707 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.888616 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-p4dw4\"" Jul 12 15:24:59.889316 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.889133 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 12 15:24:59.889316 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.889205 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 12 15:24:59.970924 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.970886 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.971134 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.970957 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.971134 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.970984 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971134 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:59.971007 2567 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 12 15:24:59.971134 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:24:59.971110 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls podName:5d14f178-0fd1-4143-a142-68b649c2b61b nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.471087341 +0000 UTC m=+278.179894917 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-cglmk" (UID: "5d14f178-0fd1-4143-a142-68b649c2b61b") : secret "openshift-state-metrics-tls" not found Jul 12 15:24:59.971134 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971015 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-sys\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971167 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh6wk\" (UniqueName: \"kubernetes.io/projected/56a2ec36-946b-4d54-96d9-f88cd45d1d93-kube-api-access-xh6wk\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971226 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nfw48\" (UniqueName: \"kubernetes.io/projected/49605801-07e1-4436-bd28-fa5d67213494-kube-api-access-nfw48\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971285 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971318 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.971369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971353 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971633 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971390 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.971693 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971662 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-root\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971746 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971703 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d14f178-0fd1-4143-a142-68b649c2b61b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.971746 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971733 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-wtmp\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971842 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971776 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-textfile\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971842 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971800 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-metrics-client-ca\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:24:59.971995 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971841 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49605801-07e1-4436-bd28-fa5d67213494-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.971995 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971873 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdfpz\" (UniqueName: \"kubernetes.io/projected/5d14f178-0fd1-4143-a142-68b649c2b61b-kube-api-access-tdfpz\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.971995 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.971907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:24:59.972461 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.972417 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5d14f178-0fd1-4143-a142-68b649c2b61b-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.974640 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.974617 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:24:59.982485 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:24:59.982427 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdfpz\" (UniqueName: \"kubernetes.io/projected/5d14f178-0fd1-4143-a142-68b649c2b61b-kube-api-access-tdfpz\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:25:00.073421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073235 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.073421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073317 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.073421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073349 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-root\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.073421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073399 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-wtmp\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073459 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-textfile\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073487 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-metrics-client-ca\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073544 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49605801-07e1-4436-bd28-fa5d67213494-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073586 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073670 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.073751 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073723 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074122 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073771 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-sys\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074122 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073798 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xh6wk\" (UniqueName: \"kubernetes.io/projected/56a2ec36-946b-4d54-96d9-f88cd45d1d93-kube-api-access-xh6wk\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074122 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073851 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nfw48\" (UniqueName: \"kubernetes.io/projected/49605801-07e1-4436-bd28-fa5d67213494-kube-api-access-nfw48\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.074122 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073878 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074122 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.073928 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.074376 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074188 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.074376 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074339 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-root\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074711 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074649 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.074711 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074685 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-textfile\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074902 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074809 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-wtmp\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.074902 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:00.073400 2567 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 12 15:25:00.074902 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:00.074893 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls podName:56a2ec36-946b-4d54-96d9-f88cd45d1d93 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.574873253 +0000 UTC m=+278.283680828 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls") pod "node-exporter-7j5xx" (UID: "56a2ec36-946b-4d54-96d9-f88cd45d1d93") : secret "node-exporter-tls" not found Jul 12 15:25:00.075070 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.074934 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-metrics-client-ca\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.075143 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:00.074300 2567 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 12 15:25:00.075197 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:00.075163 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls podName:49605801-07e1-4436-bd28-fa5d67213494 nodeName:}" failed. No retries permitted until 2026-07-12 15:25:00.575144981 +0000 UTC m=+278.283952552 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-ff8fp" (UID: "49605801-07e1-4436-bd28-fa5d67213494") : secret "kube-state-metrics-tls" not found Jul 12 15:25:00.075255 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.075221 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/56a2ec36-946b-4d54-96d9-f88cd45d1d93-sys\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.075255 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.075244 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/49605801-07e1-4436-bd28-fa5d67213494-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.075722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.075682 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-accelerators-collector-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.080819 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.080743 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.082184 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.082142 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.085317 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.085274 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nfw48\" (UniqueName: \"kubernetes.io/projected/49605801-07e1-4436-bd28-fa5d67213494-kube-api-access-nfw48\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.086938 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.086916 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh6wk\" (UniqueName: \"kubernetes.io/projected/56a2ec36-946b-4d54-96d9-f88cd45d1d93-kube-api-access-xh6wk\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.477023 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.476846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:25:00.480086 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.480036 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/5d14f178-0fd1-4143-a142-68b649c2b61b-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-cglmk\" (UID: \"5d14f178-0fd1-4143-a142-68b649c2b61b\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:25:00.577863 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.577828 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.578043 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.577909 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.580803 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.580773 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/56a2ec36-946b-4d54-96d9-f88cd45d1d93-node-exporter-tls\") pod \"node-exporter-7j5xx\" (UID: \"56a2ec36-946b-4d54-96d9-f88cd45d1d93\") " pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.580928 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.580909 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/49605801-07e1-4436-bd28-fa5d67213494-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-ff8fp\" (UID: \"49605801-07e1-4436-bd28-fa5d67213494\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.760495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.760409 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" Jul 12 15:25:00.777615 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.777579 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" Jul 12 15:25:00.797969 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.797941 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-7j5xx" Jul 12 15:25:00.888586 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.888557 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:25:00.909499 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.909452 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:25:00.909665 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.909625 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hhkvj\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912430 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912704 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912875 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912953 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.912964 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.913096 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.913216 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 12 15:25:00.913436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.913235 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 12 15:25:00.913995 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.913978 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 12 15:25:00.980211 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980176 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980408 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980222 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980408 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980253 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980408 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980293 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980408 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980331 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980651 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980478 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fznh7\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980651 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980510 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980651 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980542 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980651 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980576 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980651 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980631 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980877 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980877 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:00.980877 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:00.980749 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082232 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082203 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082268 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082299 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082545 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:01.082479 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle podName:7506ea25-230e-4e18-b4e2-06d345c8d81c nodeName:}" failed. No retries permitted until 2026-07-12 15:25:01.582455297 +0000 UTC m=+279.291262873 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c") : configmap references non-existent config key: ca-bundle.crt Jul 12 15:25:01.082693 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082669 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082786 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082719 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082786 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082752 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082793 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082853 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fznh7\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.082886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082881 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.083031 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082912 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.083031 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082946 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.083031 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.082976 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.084110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.083741 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.084110 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:01.083848 2567 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 12 15:25:01.084110 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:01.083899 2567 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls podName:7506ea25-230e-4e18-b4e2-06d345c8d81c nodeName:}" failed. No retries permitted until 2026-07-12 15:25:01.583882042 +0000 UTC m=+279.292689615 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c") : secret "alertmanager-main-tls" not found Jul 12 15:25:01.084110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.084100 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.086076 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.086016 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.087792 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.087751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.088579 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.088207 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.088579 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.088542 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.089026 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.088987 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.090086 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.090028 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.090279 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.090254 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.091263 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.091222 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.093848 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.093808 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fznh7\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.588253 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.588213 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.588436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.588288 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.589023 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.588993 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.591170 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.591147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:01.823573 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:01.823527 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:25:03.213301 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.213274 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-ff8fp"] Jul 12 15:25:03.233100 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:25:03.233064 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49605801_07e1_4436_bd28_fa5d67213494.slice/crio-004767ac4c90c3a4325c5dfc5772d173ff4268a5be94c9a00c6c9b640d63b97c WatchSource:0}: Error finding container 004767ac4c90c3a4325c5dfc5772d173ff4268a5be94c9a00c6c9b640d63b97c: Status 404 returned error can't find the container with id 004767ac4c90c3a4325c5dfc5772d173ff4268a5be94c9a00c6c9b640d63b97c Jul 12 15:25:03.438379 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.438345 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk"] Jul 12 15:25:03.441777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.441752 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:25:03.443154 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:25:03.443125 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5d14f178_0fd1_4143_a142_68b649c2b61b.slice/crio-79ad20ba55c8f3621dae72cd0a6df85a1da4ac7b619f04d9b66d4659b3b0d1f1 WatchSource:0}: Error finding container 79ad20ba55c8f3621dae72cd0a6df85a1da4ac7b619f04d9b66d4659b3b0d1f1: Status 404 returned error can't find the container with id 79ad20ba55c8f3621dae72cd0a6df85a1da4ac7b619f04d9b66d4659b3b0d1f1 Jul 12 15:25:03.446073 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:25:03.445646 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod544ac739_36c9_4fdb_a74a_914b77cd419b.slice/crio-5f56d5c548ae363af97a4d8cb3e46af0db1a53f0beafd7a842ba33d6a90aa513 WatchSource:0}: Error finding container 5f56d5c548ae363af97a4d8cb3e46af0db1a53f0beafd7a842ba33d6a90aa513: Status 404 returned error can't find the container with id 5f56d5c548ae363af97a4d8cb3e46af0db1a53f0beafd7a842ba33d6a90aa513 Jul 12 15:25:03.448836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.448729 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:25:03.451946 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:25:03.451403 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7506ea25_230e_4e18_b4e2_06d345c8d81c.slice/crio-8cc2f12c11d921416d93301c6a5c846c3a9f3349f26d821e59786086312bce08 WatchSource:0}: Error finding container 8cc2f12c11d921416d93301c6a5c846c3a9f3349f26d821e59786086312bce08: Status 404 returned error can't find the container with id 8cc2f12c11d921416d93301c6a5c846c3a9f3349f26d821e59786086312bce08 Jul 12 15:25:03.784755 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.784703 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" event={"ID":"5d14f178-0fd1-4143-a142-68b649c2b61b","Type":"ContainerStarted","Data":"51a89d69d72f41eb4d59d504f2329ac44709a11c5f58a5696976dc80817ac22c"} Jul 12 15:25:03.784938 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.784782 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" event={"ID":"5d14f178-0fd1-4143-a142-68b649c2b61b","Type":"ContainerStarted","Data":"16114eb82f223e2fa63de89ed7007b6c11a37845d86863d6766d78af4a57aa6c"} Jul 12 15:25:03.784938 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.784816 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" event={"ID":"5d14f178-0fd1-4143-a142-68b649c2b61b","Type":"ContainerStarted","Data":"79ad20ba55c8f3621dae72cd0a6df85a1da4ac7b619f04d9b66d4659b3b0d1f1"} Jul 12 15:25:03.787482 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.787377 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-fmjc5" event={"ID":"314047f5-49d6-4509-ab39-b49f0e4e6ca2","Type":"ContainerStarted","Data":"2ecad65975ff9a2a6778ca5a60a07bfa1d8603c9260a372a32ce6b359ff79f0e"} Jul 12 15:25:03.788678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.788495 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:25:03.792066 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.792016 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"8cc2f12c11d921416d93301c6a5c846c3a9f3349f26d821e59786086312bce08"} Jul 12 15:25:03.794252 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.794202 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bccbdd6fc-7ql29" event={"ID":"544ac739-36c9-4fdb-a74a-914b77cd419b","Type":"ContainerStarted","Data":"5f56d5c548ae363af97a4d8cb3e46af0db1a53f0beafd7a842ba33d6a90aa513"} Jul 12 15:25:03.795824 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.795799 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" event={"ID":"49605801-07e1-4436-bd28-fa5d67213494","Type":"ContainerStarted","Data":"004767ac4c90c3a4325c5dfc5772d173ff4268a5be94c9a00c6c9b640d63b97c"} Jul 12 15:25:03.797402 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.797329 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j5xx" event={"ID":"56a2ec36-946b-4d54-96d9-f88cd45d1d93","Type":"ContainerStarted","Data":"f245864340742322bfa5000258d9a59e45fa7d34e88e054611663ce4d92f303a"} Jul 12 15:25:03.804921 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.804852 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-fmjc5" Jul 12 15:25:03.806034 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:03.805984 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-fmjc5" podStartSLOduration=1.075422652 podStartE2EDuration="18.80596563s" totalImagesPullingTime="17.730542978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:03.804091802 +0000 UTC m=+281.512899397" watchObservedRunningTime="2026-07-12 15:25:03.80596563 +0000 UTC m=+281.514773226" Jul 12 15:25:04.136708 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.136622 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5"] Jul 12 15:25:04.139671 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.139640 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.145075 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.145018 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-wfh4l\"" Jul 12 15:25:04.145318 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.145302 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-70odv56rqg2bh\"" Jul 12 15:25:04.148563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.148536 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 12 15:25:04.148854 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.148826 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 12 15:25:04.149571 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.149407 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 12 15:25:04.149884 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.149868 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 12 15:25:04.152278 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.150853 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5"] Jul 12 15:25:04.212865 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.212818 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-client-certs\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213079 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.212905 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-client-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213079 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.212952 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-tls\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213079 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.212986 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-audit-log\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213079 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.213015 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213289 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.213098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-metrics-server-audit-profiles\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.213289 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.213129 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fg4f9\" (UniqueName: \"kubernetes.io/projected/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-kube-api-access-fg4f9\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314502 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314415 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-client-certs\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-client-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314546 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-tls\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314577 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-audit-log\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314601 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314661 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-metrics-server-audit-profiles\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.314700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.314691 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fg4f9\" (UniqueName: \"kubernetes.io/projected/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-kube-api-access-fg4f9\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.315967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.315834 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-audit-log\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.319295 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.319211 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.319556 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.318034 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-metrics-server-audit-profiles\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.338023 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.337949 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-tls\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.338745 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.338686 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-client-ca-bundle\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.339419 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.339368 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-secret-metrics-server-client-certs\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.342285 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.342237 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fg4f9\" (UniqueName: \"kubernetes.io/projected/2bcb06f4-dc4c-423e-80f7-9cdef0899fd4-kube-api-access-fg4f9\") pod \"metrics-server-5ccc6b58cf-zjcw5\" (UID: \"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4\") " pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.482455 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.482087 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:04.987249 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:04.986887 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5"] Jul 12 15:25:05.394010 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:05.393933 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-5xzvm" podUID="3a635a9e-3dbc-40c4-b4ca-7240719f4e5c" Jul 12 15:25:05.824028 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.823991 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:05.824426 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.824396 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" event={"ID":"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4","Type":"ContainerStarted","Data":"23883b1f731a48f4de45a0ede30979360278cd575334208a165e639119aea24d"} Jul 12 15:25:05.969115 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.967727 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:25:05.989176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.987340 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:05.991575 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.991483 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992114 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992266 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992407 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992443 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992652 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992684 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992740 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 12 15:25:05.992856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992124 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 12 15:25:05.993320 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.992940 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 12 15:25:05.994414 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.994392 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 12 15:25:05.994525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.994463 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 12 15:25:05.994808 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.994786 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 12 15:25:05.995183 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.995164 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 12 15:25:05.995461 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.995166 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-thd74\"" Jul 12 15:25:05.995461 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.995451 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b78kkg1jav4jg\"" Jul 12 15:25:05.995840 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:05.995821 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 12 15:25:06.047119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047085 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047298 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047174 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047298 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047248 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047298 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047289 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047324 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047359 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047399 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047450 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v4kqq\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047661 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047479 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047661 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047508 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047661 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047532 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047661 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047692 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047727 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047753 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047769 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047809 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047843 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.047903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047863 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.048277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.047908 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149325 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149437 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149468 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149494 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149517 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149552 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v4kqq\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149647 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149671 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149710 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149742 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149768 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149795 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149818 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149846 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149874 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149904 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.149957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.149934 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.154269 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.150996 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.154269 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.151674 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.156567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.155168 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.156567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.155574 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.156567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.156239 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.158061 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.157696 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.158854 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.158780 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.160173 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.159626 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.160173 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.160076 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.161401 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.161349 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.165343 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.165293 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.166252 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.165915 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.166252 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.166199 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.166430 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.166290 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.167466 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.167400 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.168007 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.167963 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.168192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.168172 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v4kqq\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.171723 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.171694 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.175985 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.175898 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.178716 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.178688 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.307674 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.307442 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:06.357844 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:06.357805 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:25:08.579126 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:08.579074 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:08.581937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:08.581911 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/3a635a9e-3dbc-40c4-b4ca-7240719f4e5c-metrics-tls\") pod \"dns-default-5xzvm\" (UID: \"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c\") " pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:08.827640 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:08.827607 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-ns4rd\"" Jul 12 15:25:08.836012 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:08.835936 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:09.205154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:09.204533 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:25:10.354670 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.354545 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5xzvm"] Jul 12 15:25:10.854845 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.854810 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="b05d650efcc9076034e61e29bb1c4387768b527c3116aec26a32899c2b162e84" exitCode=0 Jul 12 15:25:10.855224 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.855192 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"b05d650efcc9076034e61e29bb1c4387768b527c3116aec26a32899c2b162e84"} Jul 12 15:25:10.863368 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.861166 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" event={"ID":"2bcb06f4-dc4c-423e-80f7-9cdef0899fd4","Type":"ContainerStarted","Data":"413a664eaac794caa7f6cf6bebfbfa82eebde953120fab706fbec9c61474b2df"} Jul 12 15:25:10.867259 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.867175 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} Jul 12 15:25:10.867486 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.867458 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" exitCode=0 Jul 12 15:25:10.867731 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.867708 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"3385a6ca3e8b59d7cc0a2e9ca9ad65fae5dc651babd6edcc4e292a6288a110e0"} Jul 12 15:25:10.872439 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.871657 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bccbdd6fc-7ql29" event={"ID":"544ac739-36c9-4fdb-a74a-914b77cd419b","Type":"ContainerStarted","Data":"4872c352b61b9bed54a7937915bd4ff40edf9074e093d3601bc7b3d05d5a67d3"} Jul 12 15:25:10.879194 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.879098 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" event={"ID":"49605801-07e1-4436-bd28-fa5d67213494","Type":"ContainerStarted","Data":"a77f57c3c29993d1573030cef2ba3a1bb0d0b8edc7cbfa7b1aa796d1feaa507f"} Jul 12 15:25:10.879194 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.879142 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" event={"ID":"49605801-07e1-4436-bd28-fa5d67213494","Type":"ContainerStarted","Data":"eb28f8da22398e4b1d05c40185a0cd0c4d0316a871662e440b0674cad1ee8b54"} Jul 12 15:25:10.879194 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.879161 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" event={"ID":"49605801-07e1-4436-bd28-fa5d67213494","Type":"ContainerStarted","Data":"4e95468c7f46ca2dae2ef61acc7973af346f2afe62f60fae1ffffd4048d03010"} Jul 12 15:25:10.883095 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.882721 2567 generic.go:356] "Generic (PLEG): container finished" podID="56a2ec36-946b-4d54-96d9-f88cd45d1d93" containerID="ffba702ef5f5d9714f3dfbe7241aeb0bd0be7c9c1407dd934f31bdb713b181f9" exitCode=0 Jul 12 15:25:10.883095 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.882777 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j5xx" event={"ID":"56a2ec36-946b-4d54-96d9-f88cd45d1d93","Type":"ContainerDied","Data":"ffba702ef5f5d9714f3dfbe7241aeb0bd0be7c9c1407dd934f31bdb713b181f9"} Jul 12 15:25:10.886963 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.886929 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5xzvm" event={"ID":"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c","Type":"ContainerStarted","Data":"a04e5075e5dc4fcf783d3225f53443c001c9768a2be91b57b4c851ca1b45ea70"} Jul 12 15:25:10.888939 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.888851 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b585cd6-7fw8c" event={"ID":"51148455-5098-492c-9840-fdf52220f5ad","Type":"ContainerStarted","Data":"0070b5531cf09dda0fd69a6c52bc78b7a8039cd11c707f5124a3787822621862"} Jul 12 15:25:10.895635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.895599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" event={"ID":"5d14f178-0fd1-4143-a142-68b649c2b61b","Type":"ContainerStarted","Data":"90ff471467131bf8653e4cc309943ce44600c38de7fa12cc4f6ad61fdd971df9"} Jul 12 15:25:10.930365 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.930311 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-ff8fp" podStartSLOduration=5.291898643 podStartE2EDuration="11.930291821s" totalImagesPullingTime="6.638393178s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:10.929180494 +0000 UTC m=+288.637988086" watchObservedRunningTime="2026-07-12 15:25:10.930291821 +0000 UTC m=+288.639099416" Jul 12 15:25:10.946469 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.946418 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" podStartSLOduration=1.952083428 podStartE2EDuration="6.946404176s" totalImagesPullingTime="4.994320748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:25:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:10.944827043 +0000 UTC m=+288.653634640" watchObservedRunningTime="2026-07-12 15:25:10.946404176 +0000 UTC m=+288.655211771" Jul 12 15:25:10.988617 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:10.988547 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-bccbdd6fc-7ql29" podStartSLOduration=6.26544469 podStartE2EDuration="12.988523596s" totalImagesPullingTime="6.723078906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:10.98708015 +0000 UTC m=+288.695887745" watchObservedRunningTime="2026-07-12 15:25:10.988523596 +0000 UTC m=+288.697331192" Jul 12 15:25:11.004215 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:11.004168 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-679b585cd6-7fw8c" podStartSLOduration=1.529331321 podStartE2EDuration="16.004147083s" totalImagesPullingTime="14.474815762s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:11.002875025 +0000 UTC m=+288.711682620" watchObservedRunningTime="2026-07-12 15:25:11.004147083 +0000 UTC m=+288.712954678" Jul 12 15:25:11.021154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:11.021104 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-cglmk" podStartSLOduration=5.477054624 podStartE2EDuration="12.021084216s" totalImagesPullingTime="6.544029592s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:11.019680094 +0000 UTC m=+288.728487689" watchObservedRunningTime="2026-07-12 15:25:11.021084216 +0000 UTC m=+288.729891810" Jul 12 15:25:11.904672 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:11.904630 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j5xx" event={"ID":"56a2ec36-946b-4d54-96d9-f88cd45d1d93","Type":"ContainerStarted","Data":"bf98296cd44d0012e52437b87a22af196eeb2e60657ff4f19c11676ca66e36af"} Jul 12 15:25:11.905277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:11.904736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-7j5xx" event={"ID":"56a2ec36-946b-4d54-96d9-f88cd45d1d93","Type":"ContainerStarted","Data":"55fdebb0a74e8b4c3c416feaa06df23fc8074195670050f56507e4b2e2f9af48"} Jul 12 15:25:11.924242 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:11.924194 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-7j5xx" podStartSLOduration=8.726176114 podStartE2EDuration="12.924172828s" totalImagesPullingTime="3.978455988s" totalInitContainerRuntime="219.540726ms" isStatefulPod=true podCreationTimestamp="2026-07-12 15:24:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:11.92336228 +0000 UTC m=+289.632169874" watchObservedRunningTime="2026-07-12 15:25:11.924172828 +0000 UTC m=+289.632980399" Jul 12 15:25:14.920424 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:14.920387 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5xzvm" event={"ID":"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c","Type":"ContainerStarted","Data":"d62fa875ba1d8593fa6e0eb7a555cf11cec0ab000f584d9ca1f72f2b6ccbe472"} Jul 12 15:25:14.923102 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:14.923076 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"5689b84846e1be53d224c567698de2ce0224508a191931a1408d4dbede569257"} Jul 12 15:25:15.477474 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.477389 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:25:15.934225 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.934191 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"48d5600e40ba618b2f692e136138388e205374454dcfd3bd4f73c0e9609adfd8"} Jul 12 15:25:15.934815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.934273 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"2b8078c83ef614519082ac5ac1c0946bee0f7a82001af509b383eef40688019c"} Jul 12 15:25:15.934815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.934313 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"9840236de3279a3901ddd43a3709f06c9d59d05e606a03a861cb4d0a501a6c93"} Jul 12 15:25:15.934815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.934353 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"b22185680714c35ed76efe500a55ced2830945168bcfdd74f319b5a9a01baf5f"} Jul 12 15:25:15.937626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.937593 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} Jul 12 15:25:15.937766 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.937676 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} Jul 12 15:25:15.940352 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.940321 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5xzvm" event={"ID":"3a635a9e-3dbc-40c4-b4ca-7240719f4e5c","Type":"ContainerStarted","Data":"fcaaff91aae31e09073e473bf620b5b31cf79d4bd56fc1d39c1892b4fb4cd30c"} Jul 12 15:25:15.940497 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.940465 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:15.958080 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:15.957597 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-5xzvm" podStartSLOduration=253.783255124 podStartE2EDuration="4m17.957574663s" totalImagesPullingTime="4.174319539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:15.957067126 +0000 UTC m=+293.665874715" watchObservedRunningTime="2026-07-12 15:25:15.957574663 +0000 UTC m=+293.666382257" Jul 12 15:25:16.841211 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:25:16.841170 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-5x2w7" podUID="c18abc6a-0f18-4948-8c24-6bda26a89278" Jul 12 15:25:16.852585 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:16.852550 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:25:16.949364 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:16.949278 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerStarted","Data":"dda6124b93880b2b65ca9d5cca89ac349436b661761091b74ebec5b50987d813"} Jul 12 15:25:16.975415 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:16.975351 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.508178466 podStartE2EDuration="16.975333137s" totalImagesPullingTime="11.432746216s" totalInitContainerRuntime="34.408455ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:25:00 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:16.973582507 +0000 UTC m=+294.682390104" watchObservedRunningTime="2026-07-12 15:25:16.975333137 +0000 UTC m=+294.684140732" Jul 12 15:25:17.958035 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:17.958004 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} Jul 12 15:25:18.641291 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:18.641254 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:25:18.965733 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:18.965642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} Jul 12 15:25:18.965733 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:18.965711 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} Jul 12 15:25:18.966163 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:18.965741 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerStarted","Data":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} Jul 12 15:25:18.994927 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:18.994871 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=7.360842129 podStartE2EDuration="13.994853258s" totalImagesPullingTime="6.58924715s" totalInitContainerRuntime="44.763979ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:25:05 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:25:18.991783155 +0000 UTC m=+296.700590781" watchObservedRunningTime="2026-07-12 15:25:18.994853258 +0000 UTC m=+296.703660853" Jul 12 15:25:21.308466 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:21.308424 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:25:22.707623 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:22.707592 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:25:22.708298 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:22.708274 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:25:22.715555 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:22.715532 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:25:22.715938 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:22.715917 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:25:22.718422 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:22.718403 2567 kubelet.go:1648] "Image garbage collection succeeded" Jul 12 15:25:24.482693 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:24.482661 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:24.501210 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:24.482718 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:25.952008 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:25.951980 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5xzvm" Jul 12 15:25:29.839682 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:29.839647 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:25:32.489567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:32.489537 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5ds8_1d63f4ab-9cfd-46d1-9dc5-0da56873da41/serve-healthcheck-canary/0.log" Jul 12 15:25:35.922088 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:35.922021 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-679b585cd6-7fw8c" podUID="51148455-5098-492c-9840-fdf52220f5ad" containerName="console" containerID="cri-o://0070b5531cf09dda0fd69a6c52bc78b7a8039cd11c707f5124a3787822621862" gracePeriod=15 Jul 12 15:25:36.039461 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.039431 2567 generic.go:356] "Generic (PLEG): container finished" podID="51148455-5098-492c-9840-fdf52220f5ad" containerID="0070b5531cf09dda0fd69a6c52bc78b7a8039cd11c707f5124a3787822621862" exitCode=0 Jul 12 15:25:36.039599 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.039482 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b585cd6-7fw8c" event={"ID":"51148455-5098-492c-9840-fdf52220f5ad","Type":"ContainerDied","Data":"0070b5531cf09dda0fd69a6c52bc78b7a8039cd11c707f5124a3787822621862"} Jul 12 15:25:36.191406 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.191334 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:25:36.263968 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.263932 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nx59v\" (UniqueName: \"kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264165 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.263973 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264165 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264005 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264165 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264098 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264165 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264139 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264340 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264191 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert\") pod \"51148455-5098-492c-9840-fdf52220f5ad\" (UID: \"51148455-5098-492c-9840-fdf52220f5ad\") " Jul 12 15:25:36.264513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264472 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:36.264631 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264507 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:36.264631 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.264572 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config" (OuterVolumeSpecName: "console-config") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:36.266417 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.266392 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:36.266566 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.266548 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:36.266647 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.266579 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v" (OuterVolumeSpecName: "kube-api-access-nx59v") pod "51148455-5098-492c-9840-fdf52220f5ad" (UID: "51148455-5098-492c-9840-fdf52220f5ad"). InnerVolumeSpecName "kube-api-access-nx59v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:25:36.365559 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365516 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nx59v\" (UniqueName: \"kubernetes.io/projected/51148455-5098-492c-9840-fdf52220f5ad-kube-api-access-nx59v\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:36.365559 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365551 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-oauth-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:36.365559 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365561 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-oauth-serving-cert\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:36.365559 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365571 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:36.365815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365579 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/51148455-5098-492c-9840-fdf52220f5ad-console-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:36.365815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:36.365587 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/51148455-5098-492c-9840-fdf52220f5ad-console-serving-cert\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:37.044239 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:37.044202 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-679b585cd6-7fw8c" Jul 12 15:25:37.044644 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:37.044214 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-679b585cd6-7fw8c" event={"ID":"51148455-5098-492c-9840-fdf52220f5ad","Type":"ContainerDied","Data":"d3caaf5d98e68f4140d304df4b05c2059943a910fdbc5849b45985c18eac6e01"} Jul 12 15:25:37.044644 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:37.044341 2567 scope.go:117] "RemoveContainer" containerID="0070b5531cf09dda0fd69a6c52bc78b7a8039cd11c707f5124a3787822621862" Jul 12 15:25:37.061215 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:37.061179 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:25:37.065450 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:37.065425 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-679b585cd6-7fw8c"] Jul 12 15:25:38.845450 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:38.845419 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="51148455-5098-492c-9840-fdf52220f5ad" path="/var/lib/kubelet/pods/51148455-5098-492c-9840-fdf52220f5ad/volumes" Jul 12 15:25:41.876856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:41.876789 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-bccbdd6fc-7ql29" podUID="544ac739-36c9-4fdb-a74a-914b77cd419b" containerName="console" containerID="cri-o://4872c352b61b9bed54a7937915bd4ff40edf9074e093d3601bc7b3d05d5a67d3" gracePeriod=15 Jul 12 15:25:42.066905 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.066871 2567 generic.go:356] "Generic (PLEG): container finished" podID="544ac739-36c9-4fdb-a74a-914b77cd419b" containerID="4872c352b61b9bed54a7937915bd4ff40edf9074e093d3601bc7b3d05d5a67d3" exitCode=0 Jul 12 15:25:42.067113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.066941 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bccbdd6fc-7ql29" event={"ID":"544ac739-36c9-4fdb-a74a-914b77cd419b","Type":"ContainerDied","Data":"4872c352b61b9bed54a7937915bd4ff40edf9074e093d3601bc7b3d05d5a67d3"} Jul 12 15:25:42.169307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.169277 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:25:42.321378 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321337 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321378 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321380 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321628 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321516 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321628 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321595 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321628 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321625 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-84vk6\" (UniqueName: \"kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321803 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321640 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321803 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321668 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle\") pod \"544ac739-36c9-4fdb-a74a-914b77cd419b\" (UID: \"544ac739-36c9-4fdb-a74a-914b77cd419b\") " Jul 12 15:25:42.321803 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321721 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.321948 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.321836 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca" (OuterVolumeSpecName: "service-ca") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.322019 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.322001 2567 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-oauth-serving-cert\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.322113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.322026 2567 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.322113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.322083 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config" (OuterVolumeSpecName: "console-config") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.322113 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.322101 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:25:42.323884 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.323857 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:42.323884 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.323871 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6" (OuterVolumeSpecName: "kube-api-access-84vk6") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "kube-api-access-84vk6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:25:42.323884 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.323877 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "544ac739-36c9-4fdb-a74a-914b77cd419b" (UID: "544ac739-36c9-4fdb-a74a-914b77cd419b"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:25:42.422834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.422745 2567 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-serving-cert\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.422834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.422775 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-84vk6\" (UniqueName: \"kubernetes.io/projected/544ac739-36c9-4fdb-a74a-914b77cd419b-kube-api-access-84vk6\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.422834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.422788 2567 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-console-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.422834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.422796 2567 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/544ac739-36c9-4fdb-a74a-914b77cd419b-trusted-ca-bundle\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:42.422834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:42.422805 2567 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/544ac739-36c9-4fdb-a74a-914b77cd419b-console-oauth-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:25:43.075966 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:43.075916 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-bccbdd6fc-7ql29" Jul 12 15:25:43.076474 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:43.075911 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-bccbdd6fc-7ql29" event={"ID":"544ac739-36c9-4fdb-a74a-914b77cd419b","Type":"ContainerDied","Data":"5f56d5c548ae363af97a4d8cb3e46af0db1a53f0beafd7a842ba33d6a90aa513"} Jul 12 15:25:43.076474 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:43.076126 2567 scope.go:117] "RemoveContainer" containerID="4872c352b61b9bed54a7937915bd4ff40edf9074e093d3601bc7b3d05d5a67d3" Jul 12 15:25:43.096469 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:43.096439 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:25:43.097772 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:43.097753 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-bccbdd6fc-7ql29"] Jul 12 15:25:44.488967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:44.488931 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:44.493182 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:44.493157 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5ccc6b58cf-zjcw5" Jul 12 15:25:44.849119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:25:44.849083 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="544ac739-36c9-4fdb-a74a-914b77cd419b" path="/var/lib/kubelet/pods/544ac739-36c9-4fdb-a74a-914b77cd419b/volumes" Jul 12 15:26:06.308441 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:06.308399 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:06.329277 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:06.329244 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:07.200155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:07.200124 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:20.028567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028471 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028830 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="alertmanager" containerID="cri-o://5689b84846e1be53d224c567698de2ce0224508a191931a1408d4dbede569257" gracePeriod=120 Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028902 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-web" containerID="cri-o://9840236de3279a3901ddd43a3709f06c9d59d05e606a03a861cb4d0a501a6c93" gracePeriod=120 Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028902 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-metric" containerID="cri-o://48d5600e40ba618b2f692e136138388e205374454dcfd3bd4f73c0e9609adfd8" gracePeriod=120 Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028924 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="config-reloader" containerID="cri-o://b22185680714c35ed76efe500a55ced2830945168bcfdd74f319b5a9a01baf5f" gracePeriod=120 Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.028956 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="prom-label-proxy" containerID="cri-o://dda6124b93880b2b65ca9d5cca89ac349436b661761091b74ebec5b50987d813" gracePeriod=120 Jul 12 15:26:20.029149 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.029027 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy" containerID="cri-o://2b8078c83ef614519082ac5ac1c0946bee0f7a82001af509b383eef40688019c" gracePeriod=120 Jul 12 15:26:20.236635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236596 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="dda6124b93880b2b65ca9d5cca89ac349436b661761091b74ebec5b50987d813" exitCode=0 Jul 12 15:26:20.236635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236627 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="2b8078c83ef614519082ac5ac1c0946bee0f7a82001af509b383eef40688019c" exitCode=0 Jul 12 15:26:20.236635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236634 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="b22185680714c35ed76efe500a55ced2830945168bcfdd74f319b5a9a01baf5f" exitCode=0 Jul 12 15:26:20.236635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236640 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="5689b84846e1be53d224c567698de2ce0224508a191931a1408d4dbede569257" exitCode=0 Jul 12 15:26:20.236886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236667 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"dda6124b93880b2b65ca9d5cca89ac349436b661761091b74ebec5b50987d813"} Jul 12 15:26:20.236886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236759 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"2b8078c83ef614519082ac5ac1c0946bee0f7a82001af509b383eef40688019c"} Jul 12 15:26:20.236886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236788 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"b22185680714c35ed76efe500a55ced2830945168bcfdd74f319b5a9a01baf5f"} Jul 12 15:26:20.236886 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:20.236814 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"5689b84846e1be53d224c567698de2ce0224508a191931a1408d4dbede569257"} Jul 12 15:26:21.245576 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.245538 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="48d5600e40ba618b2f692e136138388e205374454dcfd3bd4f73c0e9609adfd8" exitCode=0 Jul 12 15:26:21.245576 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.245572 2567 generic.go:356] "Generic (PLEG): container finished" podID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerID="9840236de3279a3901ddd43a3709f06c9d59d05e606a03a861cb4d0a501a6c93" exitCode=0 Jul 12 15:26:21.245979 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.245592 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"48d5600e40ba618b2f692e136138388e205374454dcfd3bd4f73c0e9609adfd8"} Jul 12 15:26:21.245979 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.245654 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"9840236de3279a3901ddd43a3709f06c9d59d05e606a03a861cb4d0a501a6c93"} Jul 12 15:26:21.281870 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.281846 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:26:21.376139 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376024 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376139 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376098 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376139 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376132 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376370 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376167 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376370 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376200 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376370 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376335 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376375 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376418 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376491 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376530 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376546 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376582 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376581 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:21.376932 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376613 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.376932 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.376646 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fznh7\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7\") pod \"7506ea25-230e-4e18-b4e2-06d345c8d81c\" (UID: \"7506ea25-230e-4e18-b4e2-06d345c8d81c\") " Jul 12 15:26:21.377430 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.377148 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.377430 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.377172 2567 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-alertmanager-main-db\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.378010 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.377972 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:21.379158 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.379118 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.379158 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.379150 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.379559 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.379518 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:26:21.380015 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.379981 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7" (OuterVolumeSpecName: "kube-api-access-fznh7") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "kube-api-access-fznh7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:26:21.380363 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.380334 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.380787 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.380768 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out" (OuterVolumeSpecName: "config-out") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:26:21.380787 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.380767 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.381335 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.381317 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume" (OuterVolumeSpecName: "config-volume") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.384274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.384194 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.391377 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.391358 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config" (OuterVolumeSpecName: "web-config") pod "7506ea25-230e-4e18-b4e2-06d345c8d81c" (UID: "7506ea25-230e-4e18-b4e2-06d345c8d81c"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:21.478171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478115 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478162 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-tls-assets\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478174 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478183 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-web-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478171 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478191 2567 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478200 2567 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-cluster-tls-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478209 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/7506ea25-230e-4e18-b4e2-06d345c8d81c-config-out\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478219 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478227 2567 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7506ea25-230e-4e18-b4e2-06d345c8d81c-metrics-client-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478236 2567 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/7506ea25-230e-4e18-b4e2-06d345c8d81c-secret-alertmanager-main-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:21.478457 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:21.478244 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fznh7\" (UniqueName: \"kubernetes.io/projected/7506ea25-230e-4e18-b4e2-06d345c8d81c-kube-api-access-fznh7\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:22.253276 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.253241 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"7506ea25-230e-4e18-b4e2-06d345c8d81c","Type":"ContainerDied","Data":"8cc2f12c11d921416d93301c6a5c846c3a9f3349f26d821e59786086312bce08"} Jul 12 15:26:22.253689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.253318 2567 scope.go:117] "RemoveContainer" containerID="dda6124b93880b2b65ca9d5cca89ac349436b661761091b74ebec5b50987d813" Jul 12 15:26:22.253689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.253255 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 12 15:26:22.262115 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.262089 2567 scope.go:117] "RemoveContainer" containerID="48d5600e40ba618b2f692e136138388e205374454dcfd3bd4f73c0e9609adfd8" Jul 12 15:26:22.269492 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.269477 2567 scope.go:117] "RemoveContainer" containerID="2b8078c83ef614519082ac5ac1c0946bee0f7a82001af509b383eef40688019c" Jul 12 15:26:22.276697 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.276671 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:26:22.277176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.277159 2567 scope.go:117] "RemoveContainer" containerID="9840236de3279a3901ddd43a3709f06c9d59d05e606a03a861cb4d0a501a6c93" Jul 12 15:26:22.282404 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.282370 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 12 15:26:22.284872 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.284840 2567 scope.go:117] "RemoveContainer" containerID="b22185680714c35ed76efe500a55ced2830945168bcfdd74f319b5a9a01baf5f" Jul 12 15:26:22.292620 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.292603 2567 scope.go:117] "RemoveContainer" containerID="5689b84846e1be53d224c567698de2ce0224508a191931a1408d4dbede569257" Jul 12 15:26:22.300377 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.300353 2567 scope.go:117] "RemoveContainer" containerID="b05d650efcc9076034e61e29bb1c4387768b527c3116aec26a32899c2b162e84" Jul 12 15:26:22.845493 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:22.845461 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" path="/var/lib/kubelet/pods/7506ea25-230e-4e18-b4e2-06d345c8d81c/volumes" Jul 12 15:26:24.316434 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316403 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:24.316860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316711 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="prometheus" containerID="cri-o://e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" gracePeriod=600 Jul 12 15:26:24.316860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316742 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy" containerID="cri-o://db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" gracePeriod=600 Jul 12 15:26:24.316860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316764 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="thanos-sidecar" containerID="cri-o://9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" gracePeriod=600 Jul 12 15:26:24.316860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316791 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-thanos" containerID="cri-o://2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" gracePeriod=600 Jul 12 15:26:24.316860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316798 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-web" containerID="cri-o://1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" gracePeriod=600 Jul 12 15:26:24.317098 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.316803 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="config-reloader" containerID="cri-o://e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" gracePeriod=600 Jul 12 15:26:24.553238 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.553215 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:24.606820 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.606734 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.606820 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.606778 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.606820 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.606810 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607096 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.606980 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v4kqq\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607096 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607007 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607096 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607035 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607096 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607089 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607118 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607172 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607201 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607238 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607307 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607278 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607327 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607355 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607385 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607412 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607446 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607474 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607502 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.607696 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.607533 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2\") pod \"9fe78adf-e225-4b50-a06c-eccfff6b147e\" (UID: \"9fe78adf-e225-4b50-a06c-eccfff6b147e\") " Jul 12 15:26:24.608185 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.608067 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.608506 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.608395 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.608506 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.608419 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:26:24.609191 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.608805 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.609191 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.608892 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.610082 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.609620 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq" (OuterVolumeSpecName: "kube-api-access-v4kqq") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "kube-api-access-v4kqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:26:24.610858 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.610507 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.610858 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.610649 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.610858 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.610765 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:26:24.610858 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.610825 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.611497 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.611437 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.611973 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.611931 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.612558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.612507 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config" (OuterVolumeSpecName: "config") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.612558 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.612522 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:26:24.612723 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.612631 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.613100 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.613023 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.613178 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.613155 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.613282 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.613261 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.613493 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.613477 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out" (OuterVolumeSpecName: "config-out") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:26:24.623353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.623330 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config" (OuterVolumeSpecName: "web-config") pod "9fe78adf-e225-4b50-a06c-eccfff6b147e" (UID: "9fe78adf-e225-4b50-a06c-eccfff6b147e"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:26:24.708903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708869 2567 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-web-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.708903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708898 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-trusted-ca-bundle\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.708903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708911 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708922 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-metrics-client-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708931 2567 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-tls-assets\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708939 2567 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-config\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708947 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708957 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708966 2567 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-grpc-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708974 2567 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708981 2567 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-config-out\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708990 2567 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-kube-rbac-proxy\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.708998 2567 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-thanos-prometheus-http-client-file\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709006 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709014 2567 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-metrics-client-certs\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709022 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-db\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709031 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709039 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v4kqq\" (UniqueName: \"kubernetes.io/projected/9fe78adf-e225-4b50-a06c-eccfff6b147e-kube-api-access-v4kqq\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709069 2567 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/9fe78adf-e225-4b50-a06c-eccfff6b147e-secret-prometheus-k8s-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:24.709154 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:24.709079 2567 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/9fe78adf-e225-4b50-a06c-eccfff6b147e-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:26:25.268200 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268166 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" exitCode=0 Jul 12 15:26:25.268200 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268190 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" exitCode=0 Jul 12 15:26:25.268200 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268199 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" exitCode=0 Jul 12 15:26:25.268200 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268204 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" exitCode=0 Jul 12 15:26:25.268200 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268209 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" exitCode=0 Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268214 2567 generic.go:356] "Generic (PLEG): container finished" podID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" exitCode=0 Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268252 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268249 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268339 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268381 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268400 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268416 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268460 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268534 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} Jul 12 15:26:25.268609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.268577 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"9fe78adf-e225-4b50-a06c-eccfff6b147e","Type":"ContainerDied","Data":"3385a6ca3e8b59d7cc0a2e9ca9ad65fae5dc651babd6edcc4e292a6288a110e0"} Jul 12 15:26:25.276485 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.276461 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.283633 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.283616 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.290980 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.290932 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:25.290980 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.290965 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.294430 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.294408 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:25.298462 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.298432 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.305647 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.305629 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.313207 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.313190 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.319902 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.319880 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:25.320505 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320480 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="init-config-reloader" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320507 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="init-config-reloader" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320528 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="544ac739-36c9-4fdb-a74a-914b77cd419b" containerName="console" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320538 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="544ac739-36c9-4fdb-a74a-914b77cd419b" containerName="console" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320553 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="51148455-5098-492c-9840-fdf52220f5ad" containerName="console" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320561 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="51148455-5098-492c-9840-fdf52220f5ad" containerName="console" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320573 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320582 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.320597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320599 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320608 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320622 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="prom-label-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320630 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="prom-label-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320639 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="alertmanager" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320647 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="alertmanager" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320657 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="thanos-sidecar" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320664 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="thanos-sidecar" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320672 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320681 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320692 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="init-config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320700 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="init-config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320711 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320721 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="config-reloader" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320734 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320743 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320752 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-metric" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320760 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-metric" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320771 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="prometheus" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320779 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="prometheus" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320788 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320796 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320807 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-thanos" Jul 12 15:26:25.320857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320816 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-thanos" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320896 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-metric" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320910 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="prometheus" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320919 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320930 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="kube-rbac-proxy" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320941 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="544ac739-36c9-4fdb-a74a-914b77cd419b" containerName="console" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320949 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="51148455-5098-492c-9840-fdf52220f5ad" containerName="console" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320959 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="thanos-sidecar" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320967 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="config-reloader" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320976 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-thanos" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320985 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="prom-label-proxy" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.320994 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.321003 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="alertmanager" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.321012 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7506ea25-230e-4e18-b4e2-06d345c8d81c" containerName="config-reloader" Jul 12 15:26:25.321587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.321022 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" containerName="kube-rbac-proxy-web" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.321687 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.322012 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322043 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322096 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.322357 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322378 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.322495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322393 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.322742 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.322604 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.322742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322617 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.322742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322629 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.322844 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.322827 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.322895 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322843 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.322895 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.322853 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.323085 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.323070 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.323210 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323086 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.323210 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323096 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.323312 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.323287 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.323357 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323306 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.323357 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323320 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.323545 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:26:25.323528 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.323588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323549 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.323588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323563 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.323782 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323766 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.323818 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323782 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.324000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323978 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.324000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.323998 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.324227 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324206 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.324323 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324228 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.324412 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324397 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.324450 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324412 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.324635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324614 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.324678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324636 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.324838 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324822 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.324879 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.324837 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.325025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325003 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.325081 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325028 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.325248 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325231 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.325290 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325248 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.325431 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325416 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.325488 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325431 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.325533 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325490 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.325588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325572 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.325626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325589 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.325856 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325833 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.325933 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.325859 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.326217 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326197 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.326295 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326219 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.326458 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326439 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.326526 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326459 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.326678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326659 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.326722 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326679 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.326899 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326881 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.326973 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.326900 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.327186 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327166 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.327242 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327186 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.327408 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327392 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.327451 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327408 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.327664 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327646 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.327709 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327664 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.327866 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327848 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.327866 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.327864 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.328131 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328072 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.328223 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328133 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.328341 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328321 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 12 15:26:25.328421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328359 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 12 15:26:25.328421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328390 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.328421 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328413 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.328649 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328626 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 12 15:26:25.328649 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328643 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 12 15:26:25.328772 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328748 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.328829 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328777 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.328829 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328781 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-b78kkg1jav4jg\"" Jul 12 15:26:25.328976 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328868 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 12 15:26:25.328976 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.328902 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 12 15:26:25.329155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329095 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.329155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329121 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.329155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329141 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329168 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-thd74\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329203 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329270 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329272 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329393 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329433 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.329608 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329455 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.329912 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.329660 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 12 15:26:25.330111 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330082 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.330186 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330112 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.330384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330354 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.330467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330387 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.330649 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330627 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.330700 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330652 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.330869 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330848 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.330966 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.330878 2567 scope.go:117] "RemoveContainer" containerID="2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1" Jul 12 15:26:25.331284 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331246 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1"} err="failed to get container status \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": rpc error: code = NotFound desc = could not find container \"2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1\": container with ID starting with 2141cd07683dad1ad57d9135778b2a24a901db1b254fd8d39de61f2c3165b7b1 not found: ID does not exist" Jul 12 15:26:25.331390 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331284 2567 scope.go:117] "RemoveContainer" containerID="db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656" Jul 12 15:26:25.331567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331540 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656"} err="failed to get container status \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": rpc error: code = NotFound desc = could not find container \"db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656\": container with ID starting with db943174a3aa7f13347b86fa7c54e43f345e6e203edd15beeacf209736331656 not found: ID does not exist" Jul 12 15:26:25.331654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331566 2567 scope.go:117] "RemoveContainer" containerID="1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312" Jul 12 15:26:25.331845 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331810 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312"} err="failed to get container status \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": rpc error: code = NotFound desc = could not find container \"1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312\": container with ID starting with 1f106c39f705f8899db00170d2e71974dd289b0a127227bd1e711d68a8eeb312 not found: ID does not exist" Jul 12 15:26:25.331845 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.331836 2567 scope.go:117] "RemoveContainer" containerID="9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2" Jul 12 15:26:25.332129 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332106 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2"} err="failed to get container status \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": rpc error: code = NotFound desc = could not find container \"9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2\": container with ID starting with 9695f90ee91b526ae89f896defd87b8dd84cb65f7d3ef60ecb99844be76d70c2 not found: ID does not exist" Jul 12 15:26:25.332228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332129 2567 scope.go:117] "RemoveContainer" containerID="e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165" Jul 12 15:26:25.332291 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332271 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 12 15:26:25.332447 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332421 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165"} err="failed to get container status \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": rpc error: code = NotFound desc = could not find container \"e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165\": container with ID starting with e19c2a1fa41916a96fdf5208097b7a372c593d9121cfbc6a1b3fd4a676663165 not found: ID does not exist" Jul 12 15:26:25.332521 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332450 2567 scope.go:117] "RemoveContainer" containerID="e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33" Jul 12 15:26:25.332735 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332717 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33"} err="failed to get container status \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": rpc error: code = NotFound desc = could not find container \"e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33\": container with ID starting with e0d31ad068afbed9293f015a6cffe4e7e766a89abd3e7fec71ecefda610d8d33 not found: ID does not exist" Jul 12 15:26:25.332787 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332736 2567 scope.go:117] "RemoveContainer" containerID="38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4" Jul 12 15:26:25.332967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.332950 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4"} err="failed to get container status \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": rpc error: code = NotFound desc = could not find container \"38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4\": container with ID starting with 38f437a91b2bfcefe2ce78ba6d97d6a30caa6ac301dc05ee6786b7c45eff5de4 not found: ID does not exist" Jul 12 15:26:25.334490 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.334363 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 12 15:26:25.341377 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.341347 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:25.416175 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416142 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416175 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416199 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-config-out\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416287 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416315 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416339 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416393 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416421 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416461 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-web-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416481 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416530 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416556 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416597 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416619 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416637 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fvcc\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-kube-api-access-5fvcc\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416686 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416679 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416824 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416697 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.416824 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.416758 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518065 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518001 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518065 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518043 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518122 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-web-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518146 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518167 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518190 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518218 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518239 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518338 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fvcc\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-kube-api-access-5fvcc\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518374 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518401 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518435 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518438 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518472 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518508 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518539 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-config-out\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518564 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518594 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518620 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518648 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.518937 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.518676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.521411 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521281 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.521411 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521307 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521419 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521426 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521696 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521873 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.521915 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522141 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522163 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522169 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522215 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522243 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.522407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.522368 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.523162 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.523121 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5739e66e-158e-4554-9220-31b04d16f960-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.523776 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.523757 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-web-config\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.524863 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.524843 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5739e66e-158e-4554-9220-31b04d16f960-config-out\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.524968 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.524947 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.525006 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.524966 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.525042 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.525004 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5739e66e-158e-4554-9220-31b04d16f960-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.529553 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.529537 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fvcc\" (UniqueName: \"kubernetes.io/projected/5739e66e-158e-4554-9220-31b04d16f960-kube-api-access-5fvcc\") pod \"prometheus-k8s-0\" (UID: \"5739e66e-158e-4554-9220-31b04d16f960\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.651344 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.651304 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:25.795376 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:25.795263 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 12 15:26:25.797989 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:26:25.797960 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5739e66e_158e_4554_9220_31b04d16f960.slice/crio-4a96704a5659e5133f7b5f6e010fb80be6d028e7b5f80a486814aec56e2915e9 WatchSource:0}: Error finding container 4a96704a5659e5133f7b5f6e010fb80be6d028e7b5f80a486814aec56e2915e9: Status 404 returned error can't find the container with id 4a96704a5659e5133f7b5f6e010fb80be6d028e7b5f80a486814aec56e2915e9 Jul 12 15:26:26.273927 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:26.273885 2567 generic.go:356] "Generic (PLEG): container finished" podID="5739e66e-158e-4554-9220-31b04d16f960" containerID="b7bcd9988ff6911cee427152d71e814fc296dcf3ab4ad112d8532bb29b4603b1" exitCode=0 Jul 12 15:26:26.274125 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:26.273970 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerDied","Data":"b7bcd9988ff6911cee427152d71e814fc296dcf3ab4ad112d8532bb29b4603b1"} Jul 12 15:26:26.274125 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:26.274102 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"4a96704a5659e5133f7b5f6e010fb80be6d028e7b5f80a486814aec56e2915e9"} Jul 12 15:26:26.846636 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:26.846603 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9fe78adf-e225-4b50-a06c-eccfff6b147e" path="/var/lib/kubelet/pods/9fe78adf-e225-4b50-a06c-eccfff6b147e/volumes" Jul 12 15:26:27.282546 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282463 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"b83c9cd2ebde6e7375a09aa69c9ae40e847c08582a9613b324309a958fc06f86"} Jul 12 15:26:27.282546 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282525 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"04c1bd2d5edd07a29d05c94182fa7b033a618f2b443e49a8fef29e3196c8cd1d"} Jul 12 15:26:27.282741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282550 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"20c9f3446ca4e9619c5b0550eb8c435c81ab02c553e4145865df684ba55fc078"} Jul 12 15:26:27.282741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282574 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"0aef57e096fd7c1c246c9fad3ee618e865d408d9d01b78bd8d3cd2507dea1bec"} Jul 12 15:26:27.282741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282599 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"6412bd19291a0bede407d4d42e90c1706bc90c69a3aa627dea020ab54b171d6f"} Jul 12 15:26:27.282741 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.282624 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"5739e66e-158e-4554-9220-31b04d16f960","Type":"ContainerStarted","Data":"18367ddc70d2498469efb2bfda642b703ecb0780bfe31291055961340e085038"} Jul 12 15:26:27.308499 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:27.308445 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.289546902 podStartE2EDuration="2.30842717s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.880268ms" isStatefulPod=false podCreationTimestamp="2026-07-12 15:26:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:26:27.307222749 +0000 UTC m=+365.016030365" watchObservedRunningTime="2026-07-12 15:26:27.30842717 +0000 UTC m=+365.017234763" Jul 12 15:26:30.651905 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:30.651860 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:26:36.718304 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.718263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:26:36.720682 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.720659 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/c18abc6a-0f18-4948-8c24-6bda26a89278-metrics-certs\") pod \"network-metrics-daemon-5x2w7\" (UID: \"c18abc6a-0f18-4948-8c24-6bda26a89278\") " pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:26:36.743487 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.743455 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-9ztf2\"" Jul 12 15:26:36.751523 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.751500 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5x2w7" Jul 12 15:26:36.884990 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.884959 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5x2w7"] Jul 12 15:26:36.888313 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:26:36.888276 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc18abc6a_0f18_4948_8c24_6bda26a89278.slice/crio-1b31d4897a6bf7f8c6c9621e59195bcab6084a5362eae037ee18108a8d4e4e3a WatchSource:0}: Error finding container 1b31d4897a6bf7f8c6c9621e59195bcab6084a5362eae037ee18108a8d4e4e3a: Status 404 returned error can't find the container with id 1b31d4897a6bf7f8c6c9621e59195bcab6084a5362eae037ee18108a8d4e4e3a Jul 12 15:26:36.890028 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:36.890013 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:26:37.326697 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:37.326651 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5x2w7" event={"ID":"c18abc6a-0f18-4948-8c24-6bda26a89278","Type":"ContainerStarted","Data":"1b31d4897a6bf7f8c6c9621e59195bcab6084a5362eae037ee18108a8d4e4e3a"} Jul 12 15:26:38.333356 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:38.333311 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5x2w7" event={"ID":"c18abc6a-0f18-4948-8c24-6bda26a89278","Type":"ContainerStarted","Data":"9b9a42e681e12467c5d2ee06e1622a3aca3c464395188c3111cd6ad185d811da"} Jul 12 15:26:38.333812 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:38.333375 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5x2w7" event={"ID":"c18abc6a-0f18-4948-8c24-6bda26a89278","Type":"ContainerStarted","Data":"add70ba53bdf1fc74dedf49df53ef1cf45d2a6e6de02491e9bcce980722706c8"} Jul 12 15:26:38.353819 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:26:38.353768 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5x2w7" podStartSLOduration=374.482013931 podStartE2EDuration="6m15.353748716s" totalImagesPullingTime="871.734785ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:20:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:26:38.351468347 +0000 UTC m=+376.060275942" watchObservedRunningTime="2026-07-12 15:26:38.353748716 +0000 UTC m=+376.062556310" Jul 12 15:27:25.652463 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:27:25.652409 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:27:25.668931 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:27:25.668906 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:27:26.533841 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:27:26.533811 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 12 15:28:47.963618 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.963578 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mdpv8"] Jul 12 15:28:47.966543 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.966522 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:47.969329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.969307 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 12 15:28:47.971593 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.971576 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-x7tkm\"" Jul 12 15:28:47.971709 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.971625 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 12 15:28:47.983222 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.983192 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mdpv8"] Jul 12 15:28:47.999192 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.999164 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:47.999313 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:47.999195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vzkwc\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-kube-api-access-vzkwc\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.100495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.100458 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.100495 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.100491 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vzkwc\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-kube-api-access-vzkwc\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.118338 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.118305 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.131594 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.131559 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vzkwc\" (UniqueName: \"kubernetes.io/projected/a92f7493-d373-41e6-83fa-693c031e3005-kube-api-access-vzkwc\") pod \"cert-manager-webhook-6486ffd997-mdpv8\" (UID: \"a92f7493-d373-41e6-83fa-693c031e3005\") " pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.293960 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.293876 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:48.431574 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.431550 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-mdpv8"] Jul 12 15:28:48.433900 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:28:48.433869 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda92f7493_d373_41e6_83fa_693c031e3005.slice/crio-871f0fc671bb8e91ca7525f29af0dc1708b358e6c588140dccba534fa7f0989a WatchSource:0}: Error finding container 871f0fc671bb8e91ca7525f29af0dc1708b358e6c588140dccba534fa7f0989a: Status 404 returned error can't find the container with id 871f0fc671bb8e91ca7525f29af0dc1708b358e6c588140dccba534fa7f0989a Jul 12 15:28:48.837594 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:48.837558 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" event={"ID":"a92f7493-d373-41e6-83fa-693c031e3005","Type":"ContainerStarted","Data":"871f0fc671bb8e91ca7525f29af0dc1708b358e6c588140dccba534fa7f0989a"} Jul 12 15:28:51.852203 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:51.852151 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" event={"ID":"a92f7493-d373-41e6-83fa-693c031e3005","Type":"ContainerStarted","Data":"08301ebdc5cbe9fcf8e3b929f3dbd68f818b98cca90370ead6f7d4f1b6164d78"} Jul 12 15:28:51.852584 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:51.852388 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:28:51.868971 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:51.868919 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" podStartSLOduration=1.846048631 podStartE2EDuration="4.868903306s" totalImagesPullingTime="3.022854675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:28:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:28:51.866994541 +0000 UTC m=+509.575802158" watchObservedRunningTime="2026-07-12 15:28:51.868903306 +0000 UTC m=+509.577710900" Jul 12 15:28:57.857004 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:28:57.856968 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-mdpv8" Jul 12 15:29:21.197582 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.197491 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l"] Jul 12 15:29:21.201026 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.201000 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.204286 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.204263 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 12 15:29:21.205488 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.205459 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 12 15:29:21.205605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.205484 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-vnxtz\"" Jul 12 15:29:21.205605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.205486 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 12 15:29:21.205605 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.205463 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 12 15:29:21.205901 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.205883 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 12 15:29:21.209909 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.209875 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l"] Jul 12 15:29:21.275733 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.275696 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56cec084-7880-4945-94af-0a7eac04bd17-manager-config\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.275931 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.275745 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.275931 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.275766 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.275931 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.275850 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dmhs\" (UniqueName: \"kubernetes.io/projected/56cec084-7880-4945-94af-0a7eac04bd17-kube-api-access-7dmhs\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.376911 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.376857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56cec084-7880-4945-94af-0a7eac04bd17-manager-config\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.376911 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.376920 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.377220 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.376936 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.377220 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.376961 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dmhs\" (UniqueName: \"kubernetes.io/projected/56cec084-7880-4945-94af-0a7eac04bd17-kube-api-access-7dmhs\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.377639 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.377619 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/56cec084-7880-4945-94af-0a7eac04bd17-manager-config\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.379763 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.379732 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.379857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.379771 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/56cec084-7880-4945-94af-0a7eac04bd17-metrics-cert\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.385080 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.385039 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dmhs\" (UniqueName: \"kubernetes.io/projected/56cec084-7880-4945-94af-0a7eac04bd17-kube-api-access-7dmhs\") pod \"lws-controller-manager-59c7cf6567-88g9l\" (UID: \"56cec084-7880-4945-94af-0a7eac04bd17\") " pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.513025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.512934 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:21.663127 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.663098 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l"] Jul 12 15:29:21.664764 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:29:21.664740 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56cec084_7880_4945_94af_0a7eac04bd17.slice/crio-ea45e6be3492f213495da0299043c0ed1134402e3d91fdfe03b24094405af296 WatchSource:0}: Error finding container ea45e6be3492f213495da0299043c0ed1134402e3d91fdfe03b24094405af296: Status 404 returned error can't find the container with id ea45e6be3492f213495da0299043c0ed1134402e3d91fdfe03b24094405af296 Jul 12 15:29:21.975810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:21.975776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" event={"ID":"56cec084-7880-4945-94af-0a7eac04bd17","Type":"ContainerStarted","Data":"ea45e6be3492f213495da0299043c0ed1134402e3d91fdfe03b24094405af296"} Jul 12 15:29:24.993884 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:24.993847 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" event={"ID":"56cec084-7880-4945-94af-0a7eac04bd17","Type":"ContainerStarted","Data":"ba50e264a2d04c40bcf28a1433ad7917e42aa63efc5d10e067d8da92a9a0ce58"} Jul 12 15:29:24.994311 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:24.993945 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:25.011085 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:25.011012 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" podStartSLOduration=1.492208139 podStartE2EDuration="4.010990038s" totalImagesPullingTime="2.518781899s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:29:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:29:25.009286988 +0000 UTC m=+542.718094583" watchObservedRunningTime="2026-07-12 15:29:25.010990038 +0000 UTC m=+542.719797632" Jul 12 15:29:33.496833 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.496789 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4"] Jul 12 15:29:33.500953 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.500926 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.505397 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.505371 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-4sjfz\"" Jul 12 15:29:33.505580 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.505557 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 12 15:29:33.505711 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.505423 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 12 15:29:33.505711 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.505407 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 12 15:29:33.505834 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.505531 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 12 15:29:33.516501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.516474 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4"] Jul 12 15:29:33.581463 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.581431 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.581645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.581485 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.581645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.581561 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.581645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.581599 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.581645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.581622 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpx7b\" (UniqueName: \"kubernetes.io/projected/ee1fa10f-0d01-48c3-ab56-790d581beb5e-kube-api-access-kpx7b\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682366 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682328 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpx7b\" (UniqueName: \"kubernetes.io/projected/ee1fa10f-0d01-48c3-ab56-790d581beb5e-kube-api-access-kpx7b\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682391 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682426 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682457 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682511 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.682863 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.682841 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-manifests\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.684943 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.684914 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ee1fa10f-0d01-48c3-ab56-790d581beb5e-tmp\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.685100 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.684990 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-webhook-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.685166 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.685156 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/ee1fa10f-0d01-48c3-ab56-790d581beb5e-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.695438 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.695412 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpx7b\" (UniqueName: \"kubernetes.io/projected/ee1fa10f-0d01-48c3-ab56-790d581beb5e-kube-api-access-kpx7b\") pod \"opendatahub-operator-controller-manager-7558dcb54-992w4\" (UID: \"ee1fa10f-0d01-48c3-ab56-790d581beb5e\") " pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.812022 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.811984 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:33.962116 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:33.962086 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4"] Jul 12 15:29:33.964616 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:29:33.964588 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee1fa10f_0d01_48c3_ab56_790d581beb5e.slice/crio-4e782e0a855a27d859a24699002257049fda7e8314b29c148110021afec122fd WatchSource:0}: Error finding container 4e782e0a855a27d859a24699002257049fda7e8314b29c148110021afec122fd: Status 404 returned error can't find the container with id 4e782e0a855a27d859a24699002257049fda7e8314b29c148110021afec122fd Jul 12 15:29:34.030916 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:34.030878 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" event={"ID":"ee1fa10f-0d01-48c3-ab56-790d581beb5e","Type":"ContainerStarted","Data":"4e782e0a855a27d859a24699002257049fda7e8314b29c148110021afec122fd"} Jul 12 15:29:36.000988 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:36.000957 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59c7cf6567-88g9l" Jul 12 15:29:37.048264 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:37.048218 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" event={"ID":"ee1fa10f-0d01-48c3-ab56-790d581beb5e","Type":"ContainerStarted","Data":"1fc5f0680d555180f959c44ecd0df8d83922a6128e7b2053e671f484b7f1e23e"} Jul 12 15:29:38.054520 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:38.054489 2567 generic.go:356] "Generic (PLEG): container finished" podID="ee1fa10f-0d01-48c3-ab56-790d581beb5e" containerID="1fc5f0680d555180f959c44ecd0df8d83922a6128e7b2053e671f484b7f1e23e" exitCode=0 Jul 12 15:29:38.054930 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:38.054578 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" event={"ID":"ee1fa10f-0d01-48c3-ab56-790d581beb5e","Type":"ContainerDied","Data":"1fc5f0680d555180f959c44ecd0df8d83922a6128e7b2053e671f484b7f1e23e"} Jul 12 15:29:39.060527 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:39.060490 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" event={"ID":"ee1fa10f-0d01-48c3-ab56-790d581beb5e","Type":"ContainerStarted","Data":"268f141634cde4bb128061699dd0da3b9f8234c2cbda01cba61dd7de979969b2"} Jul 12 15:29:39.060920 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:39.060577 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:39.082119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:39.082069 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" podStartSLOduration=1.90546469 podStartE2EDuration="6.08202724s" totalImagesPullingTime="3.167988477s" totalInitContainerRuntime="1.008574073s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:29:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:29:39.080080184 +0000 UTC m=+556.788887799" watchObservedRunningTime="2026-07-12 15:29:39.08202724 +0000 UTC m=+556.790834833" Jul 12 15:29:50.067159 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:50.067125 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7558dcb54-992w4" Jul 12 15:29:56.005322 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.005289 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz"] Jul 12 15:29:56.014670 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.014641 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.018416 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.018387 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 12 15:29:56.018556 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.018449 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 12 15:29:56.018556 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.018450 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-wxdvm\"" Jul 12 15:29:56.018556 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.018514 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 12 15:29:56.018718 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.018689 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 12 15:29:56.020600 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.020535 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz"] Jul 12 15:29:56.075860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.075821 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b55013-d5e3-4886-b981-c8fc69eab194-tmp\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.075860 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.075867 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqwvn\" (UniqueName: \"kubernetes.io/projected/96b55013-d5e3-4886-b981-c8fc69eab194-kube-api-access-tqwvn\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.076159 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.075973 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b55013-d5e3-4886-b981-c8fc69eab194-tls-certs\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.176767 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.176724 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b55013-d5e3-4886-b981-c8fc69eab194-tmp\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.176767 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.176772 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqwvn\" (UniqueName: \"kubernetes.io/projected/96b55013-d5e3-4886-b981-c8fc69eab194-kube-api-access-tqwvn\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.177006 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.176821 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b55013-d5e3-4886-b981-c8fc69eab194-tls-certs\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.179236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.179206 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/96b55013-d5e3-4886-b981-c8fc69eab194-tmp\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.179418 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.179402 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/96b55013-d5e3-4886-b981-c8fc69eab194-tls-certs\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.185137 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.185114 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqwvn\" (UniqueName: \"kubernetes.io/projected/96b55013-d5e3-4886-b981-c8fc69eab194-kube-api-access-tqwvn\") pod \"kube-auth-proxy-6fb64c5677-89cjz\" (UID: \"96b55013-d5e3-4886-b981-c8fc69eab194\") " pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.326498 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.326462 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" Jul 12 15:29:56.470174 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:56.470001 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz"] Jul 12 15:29:56.472809 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:29:56.472776 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b55013_d5e3_4886_b981_c8fc69eab194.slice/crio-11e5c5fc576ed1f731566a7c064865a01199366d9b34d9a62f200af4fb1143f6 WatchSource:0}: Error finding container 11e5c5fc576ed1f731566a7c064865a01199366d9b34d9a62f200af4fb1143f6: Status 404 returned error can't find the container with id 11e5c5fc576ed1f731566a7c064865a01199366d9b34d9a62f200af4fb1143f6 Jul 12 15:29:57.139742 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:29:57.139699 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" event={"ID":"96b55013-d5e3-4886-b981-c8fc69eab194","Type":"ContainerStarted","Data":"11e5c5fc576ed1f731566a7c064865a01199366d9b34d9a62f200af4fb1143f6"} Jul 12 15:30:00.155203 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:00.155162 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" event={"ID":"96b55013-d5e3-4886-b981-c8fc69eab194","Type":"ContainerStarted","Data":"3287b724a2316afe2d155f947b7faa81fb5c2bd36cda19949e3d30cd5c2913ba"} Jul 12 15:30:00.175007 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:00.174951 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-6fb64c5677-89cjz" podStartSLOduration=2.45769748 podStartE2EDuration="5.174934679s" totalImagesPullingTime="2.717237199s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:29:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:30:00.173741426 +0000 UTC m=+577.882549018" watchObservedRunningTime="2026-07-12 15:30:00.174934679 +0000 UTC m=+577.883742273" Jul 12 15:30:22.737266 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:22.737225 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:30:22.738594 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:22.738570 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:30:22.747550 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:22.747528 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:30:22.749088 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:30:22.749067 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:31:37.890197 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.890161 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx"] Jul 12 15:31:37.892576 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.892559 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:37.896329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.896302 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 12 15:31:37.896329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.896328 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 12 15:31:37.896539 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.896474 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gj7w2\"" Jul 12 15:31:37.905757 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:37.905735 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx"] Jul 12 15:31:38.011462 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.011425 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkcb6\" (UniqueName: \"kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.011640 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.011498 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.111894 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.111857 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkcb6\" (UniqueName: \"kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.112115 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.111918 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.112300 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.112270 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.128191 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.128167 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkcb6\" (UniqueName: \"kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6\") pod \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.203075 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.202972 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:38.337725 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.337695 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx"] Jul 12 15:31:38.340779 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:31:38.340752 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad2af420_7912_44f0_a0f4_2ffe6f45fe62.slice/crio-45f918d8c98b11f12a4bcbd82e17735de081a2608e70ca8819d05f018a6fc020 WatchSource:0}: Error finding container 45f918d8c98b11f12a4bcbd82e17735de081a2608e70ca8819d05f018a6fc020: Status 404 returned error can't find the container with id 45f918d8c98b11f12a4bcbd82e17735de081a2608e70ca8819d05f018a6fc020 Jul 12 15:31:38.343870 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.343854 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:31:38.585436 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:38.585396 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" event={"ID":"ad2af420-7912-44f0-a0f4-2ffe6f45fe62","Type":"ContainerStarted","Data":"45f918d8c98b11f12a4bcbd82e17735de081a2608e70ca8819d05f018a6fc020"} Jul 12 15:31:44.616503 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:44.616466 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" event={"ID":"ad2af420-7912-44f0-a0f4-2ffe6f45fe62","Type":"ContainerStarted","Data":"9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c"} Jul 12 15:31:44.616889 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:44.616607 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:44.636367 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:44.636319 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" podStartSLOduration=1.694855813 podStartE2EDuration="7.636303442s" totalImagesPullingTime="5.941447629s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:31:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:31:44.633869932 +0000 UTC m=+682.342677529" watchObservedRunningTime="2026-07-12 15:31:44.636303442 +0000 UTC m=+682.345111035" Jul 12 15:31:55.623649 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:55.623606 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:56.631554 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.631522 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz"] Jul 12 15:31:56.633946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.633929 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.645791 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.645769 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz"] Jul 12 15:31:56.667305 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.667273 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.667650 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.667630 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m64gk\" (UniqueName: \"kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.768838 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.768801 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.768838 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.768842 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m64gk\" (UniqueName: \"kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.769269 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.769249 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.777374 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.777347 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m64gk\" (UniqueName: \"kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:56.944584 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:56.944493 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:57.084878 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.084853 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz"] Jul 12 15:31:57.087468 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:31:57.087430 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5896c79d_1d0c_42e7_8253_172238ee2514.slice/crio-c7ee9919e6dddb64f58630fd8890e7e6fe90bfd6ec3636b64276894595ed331a WatchSource:0}: Error finding container c7ee9919e6dddb64f58630fd8890e7e6fe90bfd6ec3636b64276894595ed331a: Status 404 returned error can't find the container with id c7ee9919e6dddb64f58630fd8890e7e6fe90bfd6ec3636b64276894595ed331a Jul 12 15:31:57.276823 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.276738 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz"] Jul 12 15:31:57.279477 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.279450 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz"] Jul 12 15:31:57.288124 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.288101 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx"] Jul 12 15:31:57.288311 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.288290 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" containerName="manager" containerID="cri-o://9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c" gracePeriod=2 Jul 12 15:31:57.301289 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.300963 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx"] Jul 12 15:31:57.303331 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303303 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:31:57.303818 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303799 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" containerName="manager" Jul 12 15:31:57.303818 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303820 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" containerName="manager" Jul 12 15:31:57.303818 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303831 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" containerName="manager" Jul 12 15:31:57.304026 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303840 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" containerName="manager" Jul 12 15:31:57.304026 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303916 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" containerName="manager" Jul 12 15:31:57.304026 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.303932 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" containerName="manager" Jul 12 15:31:57.306328 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.306284 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.308925 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.308867 2567 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-np8qx\": pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.311888 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.311865 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:31:57.314433 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.314416 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.322204 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.322183 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:31:57.329088 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.329042 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:31:57.352415 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.352382 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.373342 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.373319 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.373438 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.373366 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.373438 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.373421 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d4z5f\" (UniqueName: \"kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.373507 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.373439 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2lhm\" (UniqueName: \"kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.383163 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.383135 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.474329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474291 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.474458 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474352 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d4z5f\" (UniqueName: \"kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.474458 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474369 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2lhm\" (UniqueName: \"kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.474458 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474431 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.474800 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474779 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.474868 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.474792 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.483341 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.483319 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d4z5f\" (UniqueName: \"kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-hmp9v\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.483464 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.483425 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2lhm\" (UniqueName: \"kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm\") pod \"kuadrant-operator-controller-manager-8684c9ff49-hvl7n\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.517568 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.517544 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.519986 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.519962 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.575606 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.575577 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkcb6\" (UniqueName: \"kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6\") pod \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " Jul 12 15:31:57.575606 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.575610 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume\") pod \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\" (UID: \"ad2af420-7912-44f0-a0f4-2ffe6f45fe62\") " Jul 12 15:31:57.576145 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.576115 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "ad2af420-7912-44f0-a0f4-2ffe6f45fe62" (UID: "ad2af420-7912-44f0-a0f4-2ffe6f45fe62"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:31:57.577945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.577925 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6" (OuterVolumeSpecName: "kube-api-access-hkcb6") pod "ad2af420-7912-44f0-a0f4-2ffe6f45fe62" (UID: "ad2af420-7912-44f0-a0f4-2ffe6f45fe62"). InnerVolumeSpecName "kube-api-access-hkcb6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:31:57.666759 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.666717 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:57.673111 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.673083 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:57.676563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.676539 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkcb6\" (UniqueName: \"kubernetes.io/projected/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-kube-api-access-hkcb6\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:31:57.676666 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.676570 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/ad2af420-7912-44f0-a0f4-2ffe6f45fe62-extensions-socket-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:31:57.677743 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.677722 2567 generic.go:356] "Generic (PLEG): container finished" podID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" containerID="9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c" exitCode=0 Jul 12 15:31:57.677802 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.677790 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.677851 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.677805 2567 scope.go:117] "RemoveContainer" containerID="9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c" Jul 12 15:31:57.680085 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.680039 2567 kuberuntime_manager.go:649] "Missing actuated resource record" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" container="manager" Jul 12 15:31:57.680307 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.680285 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.682314 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.682284 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.684260 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.684227 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:57.688297 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.688271 2567 scope.go:117] "RemoveContainer" containerID="9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c" Jul 12 15:31:57.688664 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.688643 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c\": container with ID starting with 9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c not found: ID does not exist" containerID="9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c" Jul 12 15:31:57.688765 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.688677 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c"} err="failed to get container status \"9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c\": rpc error: code = NotFound desc = could not find container \"9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c\": container with ID starting with 9bb4060ce0e492f52a5cdf78afd41a95b46de6be24945feee9878ff53e75410c not found: ID does not exist" Jul 12 15:31:57.691897 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.691849 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:57.693829 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:57.693807 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:57.832453 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.832368 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:31:57.838330 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:31:57.835762 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9483b845_5bfc_4f57_8f36_479042d704de.slice/crio-7676cf427c0ac4944de3a12e7b4553352237d50db0aa61a0f2f55b78d640ba97 WatchSource:0}: Error finding container 7676cf427c0ac4944de3a12e7b4553352237d50db0aa61a0f2f55b78d640ba97: Status 404 returned error can't find the container with id 7676cf427c0ac4944de3a12e7b4553352237d50db0aa61a0f2f55b78d640ba97 Jul 12 15:31:57.853376 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:57.853267 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:31:57.855505 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:31:57.855473 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee21e66_fa30_4bae_91f5_0fb7c4943ed5.slice/crio-10d15988f03ca2914a624588fa225f7adf2e0284ad4e8480b592074faf0d1af4 WatchSource:0}: Error finding container 10d15988f03ca2914a624588fa225f7adf2e0284ad4e8480b592074faf0d1af4: Status 404 returned error can't find the container with id 10d15988f03ca2914a624588fa225f7adf2e0284ad4e8480b592074faf0d1af4 Jul 12 15:31:58.685946 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.685905 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" event={"ID":"aee21e66-fa30-4bae-91f5-0fb7c4943ed5","Type":"ContainerStarted","Data":"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5"} Jul 12 15:31:58.686425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.685994 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" event={"ID":"aee21e66-fa30-4bae-91f5-0fb7c4943ed5","Type":"ContainerStarted","Data":"10d15988f03ca2914a624588fa225f7adf2e0284ad4e8480b592074faf0d1af4"} Jul 12 15:31:58.686425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.686029 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:31:58.687534 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.687505 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" event={"ID":"9483b845-5bfc-4f57-8f36-479042d704de","Type":"ContainerStarted","Data":"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14"} Jul 12 15:31:58.687659 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.687560 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" event={"ID":"9483b845-5bfc-4f57-8f36-479042d704de","Type":"ContainerStarted","Data":"7676cf427c0ac4944de3a12e7b4553352237d50db0aa61a0f2f55b78d640ba97"} Jul 12 15:31:58.687659 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.687591 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:31:58.692309 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.691918 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" containerName="manager" containerID="cri-o://922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7" gracePeriod=2 Jul 12 15:31:58.706108 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.706068 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" podStartSLOduration=1.7060315369999999 podStartE2EDuration="1.706031537s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:31:58.704184676 +0000 UTC m=+696.412992295" watchObservedRunningTime="2026-07-12 15:31:58.706031537 +0000 UTC m=+696.414839130" Jul 12 15:31:58.706421 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:58.706392 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:58.708401 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:58.708376 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:58.727036 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.726989 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" podStartSLOduration=1.726970214 podStartE2EDuration="1.726970214s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:31:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:31:58.725617587 +0000 UTC m=+696.434425205" watchObservedRunningTime="2026-07-12 15:31:58.726970214 +0000 UTC m=+696.435777807" Jul 12 15:31:58.728130 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:58.728103 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-np8qx\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-np8qx" Jul 12 15:31:58.730542 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:58.730520 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:58.846176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.846142 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad2af420-7912-44f0-a0f4-2ffe6f45fe62" path="/var/lib/kubelet/pods/ad2af420-7912-44f0-a0f4-2ffe6f45fe62/volumes" Jul 12 15:31:58.933984 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.933960 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:58.987861 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.987776 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume\") pod \"5896c79d-1d0c-42e7-8253-172238ee2514\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " Jul 12 15:31:58.987861 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.987825 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m64gk\" (UniqueName: \"kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk\") pod \"5896c79d-1d0c-42e7-8253-172238ee2514\" (UID: \"5896c79d-1d0c-42e7-8253-172238ee2514\") " Jul 12 15:31:58.988110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.988073 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5896c79d-1d0c-42e7-8253-172238ee2514" (UID: "5896c79d-1d0c-42e7-8253-172238ee2514"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:31:58.990263 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:58.990235 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk" (OuterVolumeSpecName: "kube-api-access-m64gk") pod "5896c79d-1d0c-42e7-8253-172238ee2514" (UID: "5896c79d-1d0c-42e7-8253-172238ee2514"). InnerVolumeSpecName "kube-api-access-m64gk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:31:59.088629 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.088585 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5896c79d-1d0c-42e7-8253-172238ee2514-extensions-socket-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:31:59.088629 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.088616 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m64gk\" (UniqueName: \"kubernetes.io/projected/5896c79d-1d0c-42e7-8253-172238ee2514-kube-api-access-m64gk\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:31:59.697338 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.697301 2567 generic.go:356] "Generic (PLEG): container finished" podID="5896c79d-1d0c-42e7-8253-172238ee2514" containerID="922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7" exitCode=2 Jul 12 15:31:59.697799 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.697345 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:59.697799 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.697397 2567 scope.go:117] "RemoveContainer" containerID="922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7" Jul 12 15:31:59.699892 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:59.699864 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:31:59.706647 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.706633 2567 scope.go:117] "RemoveContainer" containerID="922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7" Jul 12 15:31:59.706906 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:59.706889 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7\": container with ID starting with 922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7 not found: ID does not exist" containerID="922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7" Jul 12 15:31:59.706953 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:31:59.706913 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7"} err="failed to get container status \"922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7\": rpc error: code = NotFound desc = could not find container \"922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7\": container with ID starting with 922af7220472ed4b0004980990b264df867bb52c75a27dfb191e9f2fa13e57d7 not found: ID does not exist" Jul 12 15:31:59.707791 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:31:59.707767 2567 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-9vtfz\" is forbidden: User \"system:node:ip-10-0-136-30.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-136-30.ec2.internal' and this object" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-9vtfz" Jul 12 15:32:00.846502 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:00.846469 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5896c79d-1d0c-42e7-8253-172238ee2514" path="/var/lib/kubelet/pods/5896c79d-1d0c-42e7-8253-172238ee2514/volumes" Jul 12 15:32:09.700119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.700091 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:32:09.700602 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.700159 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:32:09.771342 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.771311 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:32:09.771562 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.771537 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" podUID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" containerName="manager" containerID="cri-o://2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5" gracePeriod=10 Jul 12 15:32:09.962290 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.962222 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:32:09.965762 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.965741 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:09.977911 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:09.977883 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:32:10.031036 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.031012 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:32:10.076978 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.076954 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume\") pod \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " Jul 12 15:32:10.077155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.076985 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h2lhm\" (UniqueName: \"kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm\") pod \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\" (UID: \"aee21e66-fa30-4bae-91f5-0fb7c4943ed5\") " Jul 12 15:32:10.077155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.077098 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.077155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.077132 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5fnr\" (UniqueName: \"kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.077349 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.077317 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "aee21e66-fa30-4bae-91f5-0fb7c4943ed5" (UID: "aee21e66-fa30-4bae-91f5-0fb7c4943ed5"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:32:10.079373 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.079350 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm" (OuterVolumeSpecName: "kube-api-access-h2lhm") pod "aee21e66-fa30-4bae-91f5-0fb7c4943ed5" (UID: "aee21e66-fa30-4bae-91f5-0fb7c4943ed5"). InnerVolumeSpecName "kube-api-access-h2lhm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:32:10.178330 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.178296 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5fnr\" (UniqueName: \"kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.178501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.178375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.178501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.178401 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-extensions-socket-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:32:10.178501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.178412 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h2lhm\" (UniqueName: \"kubernetes.io/projected/aee21e66-fa30-4bae-91f5-0fb7c4943ed5-kube-api-access-h2lhm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:32:10.178697 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.178681 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.190390 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.190356 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5fnr\" (UniqueName: \"kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr\") pod \"kuadrant-operator-controller-manager-86878c769f-7j4zq\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.278486 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.278409 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.416645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.416622 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:32:10.419463 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:32:10.419436 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod946251df_1d3f_4092_9393_15e46244ea5a.slice/crio-03f7bd2bf5c8fd89188b09109cc870db164131e825387982d77035f7033f766d WatchSource:0}: Error finding container 03f7bd2bf5c8fd89188b09109cc870db164131e825387982d77035f7033f766d: Status 404 returned error can't find the container with id 03f7bd2bf5c8fd89188b09109cc870db164131e825387982d77035f7033f766d Jul 12 15:32:10.754501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.754468 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" event={"ID":"946251df-1d3f-4092-9393-15e46244ea5a","Type":"ContainerStarted","Data":"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf"} Jul 12 15:32:10.754977 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.754523 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" event={"ID":"946251df-1d3f-4092-9393-15e46244ea5a","Type":"ContainerStarted","Data":"03f7bd2bf5c8fd89188b09109cc870db164131e825387982d77035f7033f766d"} Jul 12 15:32:10.754977 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.754565 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:10.755693 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.755673 2567 generic.go:356] "Generic (PLEG): container finished" podID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" containerID="2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5" exitCode=0 Jul 12 15:32:10.755796 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.755717 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" Jul 12 15:32:10.755796 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.755723 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" event={"ID":"aee21e66-fa30-4bae-91f5-0fb7c4943ed5","Type":"ContainerDied","Data":"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5"} Jul 12 15:32:10.755892 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.755820 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n" event={"ID":"aee21e66-fa30-4bae-91f5-0fb7c4943ed5","Type":"ContainerDied","Data":"10d15988f03ca2914a624588fa225f7adf2e0284ad4e8480b592074faf0d1af4"} Jul 12 15:32:10.755892 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.755844 2567 scope.go:117] "RemoveContainer" containerID="2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5" Jul 12 15:32:10.764829 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.764813 2567 scope.go:117] "RemoveContainer" containerID="2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5" Jul 12 15:32:10.765108 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:32:10.765088 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5\": container with ID starting with 2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5 not found: ID does not exist" containerID="2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5" Jul 12 15:32:10.765202 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.765113 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5"} err="failed to get container status \"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5\": rpc error: code = NotFound desc = could not find container \"2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5\": container with ID starting with 2fa1d4713aa2e2d9ef29602856f7587d66ce7b6401fcd94aaad56fa4a937fad5 not found: ID does not exist" Jul 12 15:32:10.774135 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.774095 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" podStartSLOduration=1.7740808989999999 podStartE2EDuration="1.774080899s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:32:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:32:10.772076105 +0000 UTC m=+708.480883696" watchObservedRunningTime="2026-07-12 15:32:10.774080899 +0000 UTC m=+708.482888506" Jul 12 15:32:10.788530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.788506 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:32:10.793820 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.793798 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-hvl7n"] Jul 12 15:32:10.846471 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:10.846442 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" path="/var/lib/kubelet/pods/aee21e66-fa30-4bae-91f5-0fb7c4943ed5/volumes" Jul 12 15:32:21.763151 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:21.763043 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:32:21.811584 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:21.811550 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:32:21.812095 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:21.812019 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" podUID="9483b845-5bfc-4f57-8f36-479042d704de" containerName="manager" containerID="cri-o://58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14" gracePeriod=10 Jul 12 15:32:22.070848 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.070818 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:32:22.185089 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.185031 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume\") pod \"9483b845-5bfc-4f57-8f36-479042d704de\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " Jul 12 15:32:22.185089 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.185094 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d4z5f\" (UniqueName: \"kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f\") pod \"9483b845-5bfc-4f57-8f36-479042d704de\" (UID: \"9483b845-5bfc-4f57-8f36-479042d704de\") " Jul 12 15:32:22.185401 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.185377 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9483b845-5bfc-4f57-8f36-479042d704de" (UID: "9483b845-5bfc-4f57-8f36-479042d704de"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:32:22.187345 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.187323 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f" (OuterVolumeSpecName: "kube-api-access-d4z5f") pod "9483b845-5bfc-4f57-8f36-479042d704de" (UID: "9483b845-5bfc-4f57-8f36-479042d704de"). InnerVolumeSpecName "kube-api-access-d4z5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:32:22.286310 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.286273 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d4z5f\" (UniqueName: \"kubernetes.io/projected/9483b845-5bfc-4f57-8f36-479042d704de-kube-api-access-d4z5f\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:32:22.286310 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.286306 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9483b845-5bfc-4f57-8f36-479042d704de-extensions-socket-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:32:22.817203 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.817163 2567 generic.go:356] "Generic (PLEG): container finished" podID="9483b845-5bfc-4f57-8f36-479042d704de" containerID="58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14" exitCode=0 Jul 12 15:32:22.817597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.817213 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" Jul 12 15:32:22.817597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.817245 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" event={"ID":"9483b845-5bfc-4f57-8f36-479042d704de","Type":"ContainerDied","Data":"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14"} Jul 12 15:32:22.817597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.817319 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v" event={"ID":"9483b845-5bfc-4f57-8f36-479042d704de","Type":"ContainerDied","Data":"7676cf427c0ac4944de3a12e7b4553352237d50db0aa61a0f2f55b78d640ba97"} Jul 12 15:32:22.817597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.817344 2567 scope.go:117] "RemoveContainer" containerID="58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14" Jul 12 15:32:22.826384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.826363 2567 scope.go:117] "RemoveContainer" containerID="58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14" Jul 12 15:32:22.826646 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:32:22.826627 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14\": container with ID starting with 58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14 not found: ID does not exist" containerID="58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14" Jul 12 15:32:22.826694 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.826657 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14"} err="failed to get container status \"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14\": rpc error: code = NotFound desc = could not find container \"58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14\": container with ID starting with 58b55e1c47d8b98122fa4375f33b06506288eb9cf1f2f6d9ea7393f14315ed14 not found: ID does not exist" Jul 12 15:32:22.838874 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.838847 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:32:22.846297 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:22.846272 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-hmp9v"] Jul 12 15:32:24.845272 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:24.845241 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9483b845-5bfc-4f57-8f36-479042d704de" path="/var/lib/kubelet/pods/9483b845-5bfc-4f57-8f36-479042d704de/volumes" Jul 12 15:32:40.708210 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708173 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708504 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" containerName="manager" Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708516 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" containerName="manager" Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708526 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9483b845-5bfc-4f57-8f36-479042d704de" containerName="manager" Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708532 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9483b845-5bfc-4f57-8f36-479042d704de" containerName="manager" Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708599 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee21e66-fa30-4bae-91f5-0fb7c4943ed5" containerName="manager" Jul 12 15:32:40.708634 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.708610 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9483b845-5bfc-4f57-8f36-479042d704de" containerName="manager" Jul 12 15:32:40.711958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.711934 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:40.714671 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.714650 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5vnbc\"" Jul 12 15:32:40.718933 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.718790 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:40.738689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.738620 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7xjg\" (UniqueName: \"kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg\") pod \"authorino-58cd955675-kcn4b\" (UID: \"4789360b-a236-462f-8a19-105e2590b04b\") " pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:40.839815 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.839781 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7xjg\" (UniqueName: \"kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg\") pod \"authorino-58cd955675-kcn4b\" (UID: \"4789360b-a236-462f-8a19-105e2590b04b\") " pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:40.849036 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:40.849002 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7xjg\" (UniqueName: \"kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg\") pod \"authorino-58cd955675-kcn4b\" (UID: \"4789360b-a236-462f-8a19-105e2590b04b\") " pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:41.021455 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:41.021370 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:41.153873 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:41.153793 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:41.156367 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:32:41.156343 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4789360b_a236_462f_8a19_105e2590b04b.slice/crio-03972b65fb81da47a0cd8dd321d432b39c5c59d47515657eba06422234c49de4 WatchSource:0}: Error finding container 03972b65fb81da47a0cd8dd321d432b39c5c59d47515657eba06422234c49de4: Status 404 returned error can't find the container with id 03972b65fb81da47a0cd8dd321d432b39c5c59d47515657eba06422234c49de4 Jul 12 15:32:41.904477 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:41.904437 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kcn4b" event={"ID":"4789360b-a236-462f-8a19-105e2590b04b","Type":"ContainerStarted","Data":"03972b65fb81da47a0cd8dd321d432b39c5c59d47515657eba06422234c49de4"} Jul 12 15:32:44.920840 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:44.920804 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kcn4b" event={"ID":"4789360b-a236-462f-8a19-105e2590b04b","Type":"ContainerStarted","Data":"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f"} Jul 12 15:32:44.936152 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:44.936093 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kcn4b" podStartSLOduration=1.737246726 podStartE2EDuration="4.936077615s" totalImagesPullingTime="3.198830889s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:32:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:32:44.934755204 +0000 UTC m=+742.643562799" watchObservedRunningTime="2026-07-12 15:32:44.936077615 +0000 UTC m=+742.644885208" Jul 12 15:32:46.094759 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:46.094719 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:46.929404 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:46.929336 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kcn4b" podUID="4789360b-a236-462f-8a19-105e2590b04b" containerName="authorino" containerID="cri-o://b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f" gracePeriod=30 Jul 12 15:32:47.179525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.179464 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:47.195591 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.195564 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h7xjg\" (UniqueName: \"kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg\") pod \"4789360b-a236-462f-8a19-105e2590b04b\" (UID: \"4789360b-a236-462f-8a19-105e2590b04b\") " Jul 12 15:32:47.198158 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.198125 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg" (OuterVolumeSpecName: "kube-api-access-h7xjg") pod "4789360b-a236-462f-8a19-105e2590b04b" (UID: "4789360b-a236-462f-8a19-105e2590b04b"). InnerVolumeSpecName "kube-api-access-h7xjg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:32:47.297102 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.297040 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h7xjg\" (UniqueName: \"kubernetes.io/projected/4789360b-a236-462f-8a19-105e2590b04b-kube-api-access-h7xjg\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:32:47.934750 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.934719 2567 generic.go:356] "Generic (PLEG): container finished" podID="4789360b-a236-462f-8a19-105e2590b04b" containerID="b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f" exitCode=0 Jul 12 15:32:47.934947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.934762 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kcn4b" Jul 12 15:32:47.934947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.934789 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kcn4b" event={"ID":"4789360b-a236-462f-8a19-105e2590b04b","Type":"ContainerDied","Data":"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f"} Jul 12 15:32:47.934947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.934826 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kcn4b" event={"ID":"4789360b-a236-462f-8a19-105e2590b04b","Type":"ContainerDied","Data":"03972b65fb81da47a0cd8dd321d432b39c5c59d47515657eba06422234c49de4"} Jul 12 15:32:47.934947 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.934843 2567 scope.go:117] "RemoveContainer" containerID="b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f" Jul 12 15:32:47.943439 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.943420 2567 scope.go:117] "RemoveContainer" containerID="b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f" Jul 12 15:32:47.943684 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:32:47.943667 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f\": container with ID starting with b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f not found: ID does not exist" containerID="b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f" Jul 12 15:32:47.943747 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.943692 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f"} err="failed to get container status \"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f\": rpc error: code = NotFound desc = could not find container \"b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f\": container with ID starting with b15a01eae445f87820d65187d04ae584cf052aabc15eeead5ad6a6087ad5724f not found: ID does not exist" Jul 12 15:32:47.954155 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.954124 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:47.956549 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:47.956524 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kcn4b"] Jul 12 15:32:48.844620 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:48.844584 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4789360b-a236-462f-8a19-105e2590b04b" path="/var/lib/kubelet/pods/4789360b-a236-462f-8a19-105e2590b04b/volumes" Jul 12 15:32:59.642260 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.642226 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-qbdk6"] Jul 12 15:32:59.642958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.642751 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4789360b-a236-462f-8a19-105e2590b04b" containerName="authorino" Jul 12 15:32:59.642958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.642769 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="4789360b-a236-462f-8a19-105e2590b04b" containerName="authorino" Jul 12 15:32:59.642958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.642851 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="4789360b-a236-462f-8a19-105e2590b04b" containerName="authorino" Jul 12 15:32:59.647020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.647002 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.649992 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.649973 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xlnhv\"" Jul 12 15:32:59.650415 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.650396 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 12 15:32:59.653162 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.653136 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qbdk6"] Jul 12 15:32:59.710490 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.710454 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4xtvh\" (UniqueName: \"kubernetes.io/projected/4039edfc-5466-49ed-807c-90f8215510d0-kube-api-access-4xtvh\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.710658 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.710552 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4039edfc-5466-49ed-807c-90f8215510d0-data\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.811654 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.811619 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4039edfc-5466-49ed-807c-90f8215510d0-data\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.811831 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.811690 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4xtvh\" (UniqueName: \"kubernetes.io/projected/4039edfc-5466-49ed-807c-90f8215510d0-kube-api-access-4xtvh\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.812028 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.812008 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/4039edfc-5466-49ed-807c-90f8215510d0-data\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.821385 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.821354 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4xtvh\" (UniqueName: \"kubernetes.io/projected/4039edfc-5466-49ed-807c-90f8215510d0-kube-api-access-4xtvh\") pod \"postgres-69fb4594cf-qbdk6\" (UID: \"4039edfc-5466-49ed-807c-90f8215510d0\") " pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:32:59.958329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:32:59.958227 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:33:00.099782 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:00.099755 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-qbdk6"] Jul 12 15:33:00.100802 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:33:00.100776 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4039edfc_5466_49ed_807c_90f8215510d0.slice/crio-6e723d30fe2c6b00aecea1f56bd6706fd8901f0002b642362a1f19e5a03829af WatchSource:0}: Error finding container 6e723d30fe2c6b00aecea1f56bd6706fd8901f0002b642362a1f19e5a03829af: Status 404 returned error can't find the container with id 6e723d30fe2c6b00aecea1f56bd6706fd8901f0002b642362a1f19e5a03829af Jul 12 15:33:01.005469 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:01.005431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qbdk6" event={"ID":"4039edfc-5466-49ed-807c-90f8215510d0","Type":"ContainerStarted","Data":"6e723d30fe2c6b00aecea1f56bd6706fd8901f0002b642362a1f19e5a03829af"} Jul 12 15:33:06.031391 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:06.031348 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-qbdk6" event={"ID":"4039edfc-5466-49ed-807c-90f8215510d0","Type":"ContainerStarted","Data":"5f7525ee989a5198c536e3f14d940f1866286a443ee409efd9cb9a3dff2187eb"} Jul 12 15:33:06.031794 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:06.031471 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:33:06.047712 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:06.047672 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-qbdk6" podStartSLOduration=2.104998132 podStartE2EDuration="7.047659196s" totalImagesPullingTime="4.942661064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:32:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:33:06.047598664 +0000 UTC m=+763.756406258" watchObservedRunningTime="2026-07-12 15:33:06.047659196 +0000 UTC m=+763.756466812" Jul 12 15:33:12.068967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.068937 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-qbdk6" Jul 12 15:33:12.860188 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.860158 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dcdd966c8-58j6k"] Jul 12 15:33:12.862645 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.862628 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:12.865571 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.865552 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-5vnbc\"" Jul 12 15:33:12.872684 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.872660 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dcdd966c8-58j6k"] Jul 12 15:33:12.972547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:12.972516 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dcdd966c8-58j6k"] Jul 12 15:33:12.972810 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:33:12.972791 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-p7t9t], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-6dcdd966c8-58j6k" podUID="d914ed32-ae67-470f-a0b0-86634dc7fa92" Jul 12 15:33:13.001261 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.001230 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:33:13.003620 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.003600 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.007011 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.006995 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 12 15:33:13.013663 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.013628 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:33:13.031217 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.031195 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p7t9t\" (UniqueName: \"kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t\") pod \"authorino-6dcdd966c8-58j6k\" (UID: \"d914ed32-ae67-470f-a0b0-86634dc7fa92\") " pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:13.063784 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.063760 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:13.069265 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.069246 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:13.132198 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.132105 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p7t9t\" (UniqueName: \"kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t\") pod \"authorino-6dcdd966c8-58j6k\" (UID: \"d914ed32-ae67-470f-a0b0-86634dc7fa92\") " pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:13.132198 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.132144 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.132198 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.132177 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz64x\" (UniqueName: \"kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.140882 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.140860 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p7t9t\" (UniqueName: \"kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t\") pod \"authorino-6dcdd966c8-58j6k\" (UID: \"d914ed32-ae67-470f-a0b0-86634dc7fa92\") " pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:13.233260 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.233230 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p7t9t\" (UniqueName: \"kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t\") pod \"d914ed32-ae67-470f-a0b0-86634dc7fa92\" (UID: \"d914ed32-ae67-470f-a0b0-86634dc7fa92\") " Jul 12 15:33:13.233418 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.233370 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz64x\" (UniqueName: \"kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.233486 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.233434 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.235439 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.235414 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t" (OuterVolumeSpecName: "kube-api-access-p7t9t") pod "d914ed32-ae67-470f-a0b0-86634dc7fa92" (UID: "d914ed32-ae67-470f-a0b0-86634dc7fa92"). InnerVolumeSpecName "kube-api-access-p7t9t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:33:13.235782 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.235765 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.249170 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.249151 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz64x\" (UniqueName: \"kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x\") pod \"authorino-56d4f949bb-b966b\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.312099 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.312069 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:33:13.334402 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.334379 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p7t9t\" (UniqueName: \"kubernetes.io/projected/d914ed32-ae67-470f-a0b0-86634dc7fa92-kube-api-access-p7t9t\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:33:13.449451 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:13.449422 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:33:13.451766 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:33:13.451738 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod28c46cb8_9b75_4745_9072_09e51319aca9.slice/crio-fdac53c09424e6bcaf02d7f99c4b23264a03a89776d46da390604fabf1c7b67c WatchSource:0}: Error finding container fdac53c09424e6bcaf02d7f99c4b23264a03a89776d46da390604fabf1c7b67c: Status 404 returned error can't find the container with id fdac53c09424e6bcaf02d7f99c4b23264a03a89776d46da390604fabf1c7b67c Jul 12 15:33:14.070369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.070342 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dcdd966c8-58j6k" Jul 12 15:33:14.070790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.070337 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d4f949bb-b966b" event={"ID":"28c46cb8-9b75-4745-9072-09e51319aca9","Type":"ContainerStarted","Data":"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a"} Jul 12 15:33:14.070790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.070466 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d4f949bb-b966b" event={"ID":"28c46cb8-9b75-4745-9072-09e51319aca9","Type":"ContainerStarted","Data":"fdac53c09424e6bcaf02d7f99c4b23264a03a89776d46da390604fabf1c7b67c"} Jul 12 15:33:14.087195 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.087151 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-56d4f949bb-b966b" podStartSLOduration=1.799172641 podStartE2EDuration="2.08713575s" totalImagesPullingTime="287.963109ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:33:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:33:14.08603302 +0000 UTC m=+771.794840615" watchObservedRunningTime="2026-07-12 15:33:14.08713575 +0000 UTC m=+771.795943344" Jul 12 15:33:14.118359 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.118330 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dcdd966c8-58j6k"] Jul 12 15:33:14.128912 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.128869 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dcdd966c8-58j6k"] Jul 12 15:33:14.845981 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:33:14.845938 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d914ed32-ae67-470f-a0b0-86634dc7fa92" path="/var/lib/kubelet/pods/d914ed32-ae67-470f-a0b0-86634dc7fa92/volumes" Jul 12 15:34:25.692554 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.692521 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr"] Jul 12 15:34:25.696022 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.695998 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.698630 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.698608 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 12 15:34:25.698736 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.698640 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 12 15:34:25.699924 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.699904 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 12 15:34:25.700023 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.699953 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-nqvwk\"" Jul 12 15:34:25.705664 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.705643 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr"] Jul 12 15:34:25.777342 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777309 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.777540 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777349 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.777540 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777375 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.777540 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777487 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7945r\" (UniqueName: \"kubernetes.io/projected/839c3860-eb8b-426a-b6ee-18f25b3e9e88-kube-api-access-7945r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.777540 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777524 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.777745 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.777552 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.878820 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.878790 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7945r\" (UniqueName: \"kubernetes.io/projected/839c3860-eb8b-426a-b6ee-18f25b3e9e88-kube-api-access-7945r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.878825 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879000 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.878847 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879029 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879098 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879138 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879186 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879274 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879264 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.879353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.879338 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.881468 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.881446 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/839c3860-eb8b-426a-b6ee-18f25b3e9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.881803 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.881785 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/839c3860-eb8b-426a-b6ee-18f25b3e9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:25.887801 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:25.887774 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7945r\" (UniqueName: \"kubernetes.io/projected/839c3860-eb8b-426a-b6ee-18f25b3e9e88-kube-api-access-7945r\") pod \"e2e-distinct-simulated-kserve-59975d69f9-8bjcr\" (UID: \"839c3860-eb8b-426a-b6ee-18f25b3e9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:26.006588 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:26.006504 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:26.148221 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:26.148196 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr"] Jul 12 15:34:26.151564 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:34:26.151527 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod839c3860_eb8b_426a_b6ee_18f25b3e9e88.slice/crio-84bdc42e29b282979413cf5f65d27bf37811b0df6b902158797ad5ac486bcccc WatchSource:0}: Error finding container 84bdc42e29b282979413cf5f65d27bf37811b0df6b902158797ad5ac486bcccc: Status 404 returned error can't find the container with id 84bdc42e29b282979413cf5f65d27bf37811b0df6b902158797ad5ac486bcccc Jul 12 15:34:26.405866 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:26.405814 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" event={"ID":"839c3860-eb8b-426a-b6ee-18f25b3e9e88","Type":"ContainerStarted","Data":"84bdc42e29b282979413cf5f65d27bf37811b0df6b902158797ad5ac486bcccc"} Jul 12 15:34:27.412680 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:27.412642 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" event={"ID":"839c3860-eb8b-426a-b6ee-18f25b3e9e88","Type":"ContainerStarted","Data":"67b250bcdb1aa9dcabbba7579fc03f4bb6e60a59135cfb1cdb39a230928f479d"} Jul 12 15:34:27.432511 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:27.432461 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" podStartSLOduration=1.341513958 podStartE2EDuration="2.432444731s" totalImagesPullingTime="1.090930773s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:34:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:34:27.430746156 +0000 UTC m=+845.139553751" watchObservedRunningTime="2026-07-12 15:34:27.432444731 +0000 UTC m=+845.141252324" Jul 12 15:34:36.007223 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:36.007182 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:36.007812 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:36.007262 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:36.020667 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:36.020639 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:36.464811 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:36.464777 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-8bjcr" Jul 12 15:34:50.095790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.095755 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw"] Jul 12 15:34:50.100250 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.100228 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.103141 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.103121 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 12 15:34:50.111685 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.111638 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw"] Jul 12 15:34:50.205511 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205475 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1725fb5c-fddc-4f0e-9729-c741adf1c535-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.205678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.205678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205608 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.205678 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205642 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.205796 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205701 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6btq\" (UniqueName: \"kubernetes.io/projected/1725fb5c-fddc-4f0e-9729-c741adf1c535-kube-api-access-j6btq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.205796 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.205768 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306315 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306378 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306412 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j6btq\" (UniqueName: \"kubernetes.io/projected/1725fb5c-fddc-4f0e-9729-c741adf1c535-kube-api-access-j6btq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306455 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306509 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1725fb5c-fddc-4f0e-9729-c741adf1c535-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306626 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306550 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306774 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306803 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.306950 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.306862 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.308835 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.308814 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1725fb5c-fddc-4f0e-9729-c741adf1c535-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.309206 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.309185 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1725fb5c-fddc-4f0e-9729-c741adf1c535-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.314359 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.314338 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6btq\" (UniqueName: \"kubernetes.io/projected/1725fb5c-fddc-4f0e-9729-c741adf1c535-kube-api-access-j6btq\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-v24cw\" (UID: \"1725fb5c-fddc-4f0e-9729-c741adf1c535\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.411532 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.411449 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:34:50.545529 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:50.545506 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw"] Jul 12 15:34:50.548201 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:34:50.548171 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1725fb5c_fddc_4f0e_9729_c741adf1c535.slice/crio-875873872da5ca2fa732c04e9672ec8f7da07a60e8c06106ae29e993026a594e WatchSource:0}: Error finding container 875873872da5ca2fa732c04e9672ec8f7da07a60e8c06106ae29e993026a594e: Status 404 returned error can't find the container with id 875873872da5ca2fa732c04e9672ec8f7da07a60e8c06106ae29e993026a594e Jul 12 15:34:51.524960 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:51.524920 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" event={"ID":"1725fb5c-fddc-4f0e-9729-c741adf1c535","Type":"ContainerStarted","Data":"69edc33c9c1cafac50aa6ea4aa00972817b53403efd96e9007aefdd3537f7b73"} Jul 12 15:34:51.525429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:51.524974 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" event={"ID":"1725fb5c-fddc-4f0e-9729-c741adf1c535","Type":"ContainerStarted","Data":"875873872da5ca2fa732c04e9672ec8f7da07a60e8c06106ae29e993026a594e"} Jul 12 15:34:51.544300 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:34:51.544242 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" podStartSLOduration=1.3823697369999999 podStartE2EDuration="1.544226664s" totalImagesPullingTime="161.856927ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:34:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:34:51.542211762 +0000 UTC m=+869.251019383" watchObservedRunningTime="2026-07-12 15:34:51.544226664 +0000 UTC m=+869.253034257" Jul 12 15:35:00.412079 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:00.412032 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:35:00.412555 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:00.412106 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:35:00.424727 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:00.424699 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:35:00.579901 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:00.579848 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-v24cw" Jul 12 15:35:22.769243 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:22.769210 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:35:22.772074 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:22.772033 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:35:22.776384 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:22.776366 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:35:22.779090 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:22.779074 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:35:48.985211 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:48.985178 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-df97db75-wqb4m"] Jul 12 15:35:48.988623 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:48.988604 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:48.997987 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:48.997964 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-df97db75-wqb4m"] Jul 12 15:35:49.042563 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.042525 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r2h94\" (UniqueName: \"kubernetes.io/projected/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-kube-api-access-r2h94\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.042727 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.042600 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-tls-cert\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.143316 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.143281 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r2h94\" (UniqueName: \"kubernetes.io/projected/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-kube-api-access-r2h94\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.143501 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.143348 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-tls-cert\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.145869 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.145850 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-tls-cert\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.152879 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.152855 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r2h94\" (UniqueName: \"kubernetes.io/projected/c99e4b48-c433-4f70-b2e3-df9ddcd1f859-kube-api-access-r2h94\") pod \"authorino-df97db75-wqb4m\" (UID: \"c99e4b48-c433-4f70-b2e3-df9ddcd1f859\") " pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.300110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.300001 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-df97db75-wqb4m" Jul 12 15:35:49.431142 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.431110 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-df97db75-wqb4m"] Jul 12 15:35:49.433606 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:35:49.433579 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc99e4b48_c433_4f70_b2e3_df9ddcd1f859.slice/crio-325d14bfcbe01e2ddbef7a6b8f5ee736a65cce49a23d4567bc5ebc8e46b29d44 WatchSource:0}: Error finding container 325d14bfcbe01e2ddbef7a6b8f5ee736a65cce49a23d4567bc5ebc8e46b29d44: Status 404 returned error can't find the container with id 325d14bfcbe01e2ddbef7a6b8f5ee736a65cce49a23d4567bc5ebc8e46b29d44 Jul 12 15:35:49.802490 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:49.802448 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-df97db75-wqb4m" event={"ID":"c99e4b48-c433-4f70-b2e3-df9ddcd1f859","Type":"ContainerStarted","Data":"325d14bfcbe01e2ddbef7a6b8f5ee736a65cce49a23d4567bc5ebc8e46b29d44"} Jul 12 15:35:50.810111 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:50.810070 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-df97db75-wqb4m" event={"ID":"c99e4b48-c433-4f70-b2e3-df9ddcd1f859","Type":"ContainerStarted","Data":"b08d7df27f2404994538e4a0bd8cfc185a3351ddbf8c180645a45823271938ae"} Jul 12 15:35:50.825770 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:50.825669 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-df97db75-wqb4m" podStartSLOduration=2.391109873 podStartE2EDuration="2.825647906s" totalImagesPullingTime="434.538033ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:35:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:35:50.824822873 +0000 UTC m=+928.533630488" watchObservedRunningTime="2026-07-12 15:35:50.825647906 +0000 UTC m=+928.534455511" Jul 12 15:35:50.854184 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:50.854149 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:35:50.854451 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:50.854410 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-56d4f949bb-b966b" podUID="28c46cb8-9b75-4745-9072-09e51319aca9" containerName="authorino" containerID="cri-o://fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a" gracePeriod=30 Jul 12 15:35:51.131642 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.131354 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:35:51.161338 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.161305 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz64x\" (UniqueName: \"kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x\") pod \"28c46cb8-9b75-4745-9072-09e51319aca9\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " Jul 12 15:35:51.161509 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.161406 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert\") pod \"28c46cb8-9b75-4745-9072-09e51319aca9\" (UID: \"28c46cb8-9b75-4745-9072-09e51319aca9\") " Jul 12 15:35:51.163793 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.163758 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x" (OuterVolumeSpecName: "kube-api-access-tz64x") pod "28c46cb8-9b75-4745-9072-09e51319aca9" (UID: "28c46cb8-9b75-4745-9072-09e51319aca9"). InnerVolumeSpecName "kube-api-access-tz64x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:35:51.174399 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.174366 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "28c46cb8-9b75-4745-9072-09e51319aca9" (UID: "28c46cb8-9b75-4745-9072-09e51319aca9"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:35:51.263038 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.263007 2567 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/28c46cb8-9b75-4745-9072-09e51319aca9-tls-cert\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:35:51.263038 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.263037 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz64x\" (UniqueName: \"kubernetes.io/projected/28c46cb8-9b75-4745-9072-09e51319aca9-kube-api-access-tz64x\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:35:51.816146 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.816109 2567 generic.go:356] "Generic (PLEG): container finished" podID="28c46cb8-9b75-4745-9072-09e51319aca9" containerID="fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a" exitCode=0 Jul 12 15:35:51.816680 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.816164 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-56d4f949bb-b966b" Jul 12 15:35:51.816680 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.816203 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d4f949bb-b966b" event={"ID":"28c46cb8-9b75-4745-9072-09e51319aca9","Type":"ContainerDied","Data":"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a"} Jul 12 15:35:51.816680 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.816255 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-56d4f949bb-b966b" event={"ID":"28c46cb8-9b75-4745-9072-09e51319aca9","Type":"ContainerDied","Data":"fdac53c09424e6bcaf02d7f99c4b23264a03a89776d46da390604fabf1c7b67c"} Jul 12 15:35:51.816680 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.816278 2567 scope.go:117] "RemoveContainer" containerID="fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a" Jul 12 15:35:51.825228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.825211 2567 scope.go:117] "RemoveContainer" containerID="fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a" Jul 12 15:35:51.825496 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:35:51.825479 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a\": container with ID starting with fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a not found: ID does not exist" containerID="fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a" Jul 12 15:35:51.825557 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.825504 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a"} err="failed to get container status \"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a\": rpc error: code = NotFound desc = could not find container \"fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a\": container with ID starting with fb93db964fe14b407c6a04e2440d2d5ab2447fe6c6230e7be11a48ce0eda5d9a not found: ID does not exist" Jul 12 15:35:51.861291 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.861256 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:35:51.864304 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:51.864274 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-56d4f949bb-b966b"] Jul 12 15:35:52.845569 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:35:52.845535 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28c46cb8-9b75-4745-9072-09e51319aca9" path="/var/lib/kubelet/pods/28c46cb8-9b75-4745-9072-09e51319aca9/volumes" Jul 12 15:40:22.797881 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:40:22.797854 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:40:22.803281 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:40:22.803256 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:40:22.805393 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:40:22.805369 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:40:22.809995 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:40:22.809980 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:41:48.337790 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:48.337705 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:41:48.340257 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:48.337923 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" podUID="946251df-1d3f-4092-9393-15e46244ea5a" containerName="manager" containerID="cri-o://7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf" gracePeriod=10 Jul 12 15:41:51.195489 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.195464 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:41:51.274287 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.274208 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume\") pod \"946251df-1d3f-4092-9393-15e46244ea5a\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " Jul 12 15:41:51.274446 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.274305 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5fnr\" (UniqueName: \"kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr\") pod \"946251df-1d3f-4092-9393-15e46244ea5a\" (UID: \"946251df-1d3f-4092-9393-15e46244ea5a\") " Jul 12 15:41:51.274599 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.274576 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "946251df-1d3f-4092-9393-15e46244ea5a" (UID: "946251df-1d3f-4092-9393-15e46244ea5a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:41:51.276383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.276364 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr" (OuterVolumeSpecName: "kube-api-access-d5fnr") pod "946251df-1d3f-4092-9393-15e46244ea5a" (UID: "946251df-1d3f-4092-9393-15e46244ea5a"). InnerVolumeSpecName "kube-api-access-d5fnr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:41:51.374912 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.374884 2567 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/946251df-1d3f-4092-9393-15e46244ea5a-extensions-socket-volume\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:41:51.374912 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.374909 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5fnr\" (UniqueName: \"kubernetes.io/projected/946251df-1d3f-4092-9393-15e46244ea5a-kube-api-access-d5fnr\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:41:51.499353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.499314 2567 generic.go:356] "Generic (PLEG): container finished" podID="946251df-1d3f-4092-9393-15e46244ea5a" containerID="7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf" exitCode=0 Jul 12 15:41:51.499535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.499361 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" Jul 12 15:41:51.499535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.499365 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" event={"ID":"946251df-1d3f-4092-9393-15e46244ea5a","Type":"ContainerDied","Data":"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf"} Jul 12 15:41:51.499535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.499419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq" event={"ID":"946251df-1d3f-4092-9393-15e46244ea5a","Type":"ContainerDied","Data":"03f7bd2bf5c8fd89188b09109cc870db164131e825387982d77035f7033f766d"} Jul 12 15:41:51.499535 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.499445 2567 scope.go:117] "RemoveContainer" containerID="7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf" Jul 12 15:41:51.509039 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.509020 2567 scope.go:117] "RemoveContainer" containerID="7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf" Jul 12 15:41:51.509306 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:41:51.509286 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf\": container with ID starting with 7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf not found: ID does not exist" containerID="7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf" Jul 12 15:41:51.509365 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.509316 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf"} err="failed to get container status \"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf\": rpc error: code = NotFound desc = could not find container \"7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf\": container with ID starting with 7d6342532ebe1daf7b517c50cad4e2df40dca937e6b374f8d5491c12cc50c9cf not found: ID does not exist" Jul 12 15:41:51.520309 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.520286 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:41:51.524900 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:51.524841 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-7j4zq"] Jul 12 15:41:52.846227 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:41:52.846190 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="946251df-1d3f-4092-9393-15e46244ea5a" path="/var/lib/kubelet/pods/946251df-1d3f-4092-9393-15e46244ea5a/volumes" Jul 12 15:42:49.458652 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.458573 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b"] Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.458923 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="946251df-1d3f-4092-9393-15e46244ea5a" containerName="manager" Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.458934 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="946251df-1d3f-4092-9393-15e46244ea5a" containerName="manager" Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.458943 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28c46cb8-9b75-4745-9072-09e51319aca9" containerName="authorino" Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.458948 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="28c46cb8-9b75-4745-9072-09e51319aca9" containerName="authorino" Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.459003 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="28c46cb8-9b75-4745-9072-09e51319aca9" containerName="authorino" Jul 12 15:42:49.459084 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.459010 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="946251df-1d3f-4092-9393-15e46244ea5a" containerName="manager" Jul 12 15:42:49.463069 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.463039 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.465809 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.465788 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-gj7w2\"" Jul 12 15:42:49.473644 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.473617 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b"] Jul 12 15:42:49.544304 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.544256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.544304 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.544303 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xtld\" (UniqueName: \"kubernetes.io/projected/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-kube-api-access-9xtld\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.645259 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.645222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.645259 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.645261 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9xtld\" (UniqueName: \"kubernetes.io/projected/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-kube-api-access-9xtld\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.645632 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.645611 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.653522 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.653501 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xtld\" (UniqueName: \"kubernetes.io/projected/88aa582d-fda9-4fc1-877f-0ad81b1fdef0-kube-api-access-9xtld\") pod \"kuadrant-operator-controller-manager-86878c769f-l8l2b\" (UID: \"88aa582d-fda9-4fc1-877f-0ad81b1fdef0\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.774278 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.774186 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:49.909479 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:49.909452 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b"] Jul 12 15:42:49.912151 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:42:49.912116 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88aa582d_fda9_4fc1_877f_0ad81b1fdef0.slice/crio-3600baa8c0ae6691434fc87ab71f6a655a0033d09d1b0e93e0dd32dc9d69fcfe WatchSource:0}: Error finding container 3600baa8c0ae6691434fc87ab71f6a655a0033d09d1b0e93e0dd32dc9d69fcfe: Status 404 returned error can't find the container with id 3600baa8c0ae6691434fc87ab71f6a655a0033d09d1b0e93e0dd32dc9d69fcfe Jul 12 15:42:50.763025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:50.762988 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" event={"ID":"88aa582d-fda9-4fc1-877f-0ad81b1fdef0","Type":"ContainerStarted","Data":"c4515d447f45380d5ef84552d7ab6277c0ad458a00475d7368ba2d752df78fd9"} Jul 12 15:42:50.763530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:50.763069 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" event={"ID":"88aa582d-fda9-4fc1-877f-0ad81b1fdef0","Type":"ContainerStarted","Data":"3600baa8c0ae6691434fc87ab71f6a655a0033d09d1b0e93e0dd32dc9d69fcfe"} Jul 12 15:42:50.763530 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:50.763234 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:42:50.782567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:42:50.782523 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" podStartSLOduration=1.782507721 podStartE2EDuration="1.782507721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:42:50.780339798 +0000 UTC m=+1348.489147394" watchObservedRunningTime="2026-07-12 15:42:50.782507721 +0000 UTC m=+1348.491315314" Jul 12 15:43:01.770560 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:43:01.770530 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-l8l2b" Jul 12 15:45:22.826074 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:45:22.826024 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:45:22.832329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:45:22.832306 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:45:22.833264 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:45:22.833248 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:45:22.839439 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:45:22.839420 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:49:01.713013 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.712979 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:01.716625 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.716603 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.719162 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.719137 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-6cf7acf5-serving-cert\"" Jul 12 15:49:01.719364 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.719198 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:49:01.725162 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.725142 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:01.879337 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.879296 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q287w\" (UniqueName: \"kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.879513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.879367 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.879513 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.879390 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.980785 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.980676 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.980785 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.980735 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.981029 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.980812 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q287w\" (UniqueName: \"kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.981333 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.981311 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.983261 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.983234 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:01.989320 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:01.989286 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q287w\" (UniqueName: \"kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w\") pod \"maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:02.029286 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:02.029256 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:02.167426 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:02.167400 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:02.169534 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:49:02.169509 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d33af72_8ff5_4dd0_aae8_04508fc1f210.slice/crio-7d02fd23ad1dce883fce7ea225eb7d47fc7fcac48d56f0ad8054708f8ba8a879 WatchSource:0}: Error finding container 7d02fd23ad1dce883fce7ea225eb7d47fc7fcac48d56f0ad8054708f8ba8a879: Status 404 returned error can't find the container with id 7d02fd23ad1dce883fce7ea225eb7d47fc7fcac48d56f0ad8054708f8ba8a879 Jul 12 15:49:02.170859 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:02.170841 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:49:02.518613 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:02.518568 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" event={"ID":"2d33af72-8ff5-4dd0-aae8-04508fc1f210","Type":"ContainerStarted","Data":"7d02fd23ad1dce883fce7ea225eb7d47fc7fcac48d56f0ad8054708f8ba8a879"} Jul 12 15:49:04.530873 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:04.530834 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" event={"ID":"2d33af72-8ff5-4dd0-aae8-04508fc1f210","Type":"ContainerStarted","Data":"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d"} Jul 12 15:49:04.531261 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:04.530901 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:04.547369 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:04.547328 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" podStartSLOduration=1.8058256830000001 podStartE2EDuration="3.547315584s" totalImagesPullingTime="1.741489901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:49:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:49:04.546081673 +0000 UTC m=+1722.254889292" watchObservedRunningTime="2026-07-12 15:49:04.547315584 +0000 UTC m=+1722.256123178" Jul 12 15:49:07.002776 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.002738 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:07.003284 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.003027 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" podUID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" containerName="maas-api" containerID="cri-o://95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d" gracePeriod=30 Jul 12 15:49:07.249548 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.249525 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:07.429762 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.429730 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls\") pod \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " Jul 12 15:49:07.429967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.429795 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q287w\" (UniqueName: \"kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w\") pod \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " Jul 12 15:49:07.429967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.429894 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca\") pod \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\" (UID: \"2d33af72-8ff5-4dd0-aae8-04508fc1f210\") " Jul 12 15:49:07.430299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.430275 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2d33af72-8ff5-4dd0-aae8-04508fc1f210" (UID: "2d33af72-8ff5-4dd0-aae8-04508fc1f210"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:49:07.432029 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.432001 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w" (OuterVolumeSpecName: "kube-api-access-q287w") pod "2d33af72-8ff5-4dd0-aae8-04508fc1f210" (UID: "2d33af72-8ff5-4dd0-aae8-04508fc1f210"). InnerVolumeSpecName "kube-api-access-q287w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:07.432029 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.432012 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2d33af72-8ff5-4dd0-aae8-04508fc1f210" (UID: "2d33af72-8ff5-4dd0-aae8-04508fc1f210"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:49:07.531264 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.531225 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2d33af72-8ff5-4dd0-aae8-04508fc1f210-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:07.531264 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.531253 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2d33af72-8ff5-4dd0-aae8-04508fc1f210-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:07.531264 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.531263 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q287w\" (UniqueName: \"kubernetes.io/projected/2d33af72-8ff5-4dd0-aae8-04508fc1f210-kube-api-access-q287w\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:07.544760 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.544727 2567 generic.go:356] "Generic (PLEG): container finished" podID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" containerID="95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d" exitCode=0 Jul 12 15:49:07.544910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.544770 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" Jul 12 15:49:07.544910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.544801 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" event={"ID":"2d33af72-8ff5-4dd0-aae8-04508fc1f210","Type":"ContainerDied","Data":"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d"} Jul 12 15:49:07.544910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.544876 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt" event={"ID":"2d33af72-8ff5-4dd0-aae8-04508fc1f210","Type":"ContainerDied","Data":"7d02fd23ad1dce883fce7ea225eb7d47fc7fcac48d56f0ad8054708f8ba8a879"} Jul 12 15:49:07.544910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.544904 2567 scope.go:117] "RemoveContainer" containerID="95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d" Jul 12 15:49:07.554928 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.554907 2567 scope.go:117] "RemoveContainer" containerID="95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d" Jul 12 15:49:07.555198 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:49:07.555175 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d\": container with ID starting with 95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d not found: ID does not exist" containerID="95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d" Jul 12 15:49:07.555270 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.555207 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d"} err="failed to get container status \"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d\": rpc error: code = NotFound desc = could not find container \"95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d\": container with ID starting with 95989d6cf69a297889d755bdfd912efc1d0ee48db557b2ad2e892755a824a36d not found: ID does not exist" Jul 12 15:49:07.567547 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.567523 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:07.570660 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:07.570635 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-6cf7acf5-6cc8b45b66-bmvlt"] Jul 12 15:49:08.845635 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:08.845599 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" path="/var/lib/kubelet/pods/2d33af72-8ff5-4dd0-aae8-04508fc1f210/volumes" Jul 12 15:49:14.232887 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.232842 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:14.233377 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.233361 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" containerName="maas-api" Jul 12 15:49:14.233377 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.233377 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" containerName="maas-api" Jul 12 15:49:14.233528 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.233448 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d33af72-8ff5-4dd0-aae8-04508fc1f210" containerName="maas-api" Jul 12 15:49:14.237644 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.237620 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.240517 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.240408 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-5d5046a9-serving-cert\"" Jul 12 15:49:14.240517 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.240438 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:49:14.246595 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.245761 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:14.389227 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.389190 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xb85\" (UniqueName: \"kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.389397 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.389243 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.389397 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.389307 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.489945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.489860 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.489945 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.489917 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xb85\" (UniqueName: \"kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.490212 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.489954 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.490581 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.490557 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.492407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.492386 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.498726 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.498706 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xb85\" (UniqueName: \"kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85\") pod \"maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.549655 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.549625 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:14.688854 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:14.688823 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:14.692635 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:49:14.692608 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98bb3a5f_ee94_4775_8f3d_affb7244c0cc.slice/crio-e8f4e371953b79d1f85525d7c2d1d301e9214350d4ce69fd70791f8d4434cf71 WatchSource:0}: Error finding container e8f4e371953b79d1f85525d7c2d1d301e9214350d4ce69fd70791f8d4434cf71: Status 404 returned error can't find the container with id e8f4e371953b79d1f85525d7c2d1d301e9214350d4ce69fd70791f8d4434cf71 Jul 12 15:49:15.590043 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:15.590002 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" event={"ID":"98bb3a5f-ee94-4775-8f3d-affb7244c0cc","Type":"ContainerStarted","Data":"ff2a00ac50cfeaef6d412000a790d29772ea3767e9b6d8e74b80c5a748605811"} Jul 12 15:49:15.590445 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:15.590195 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:15.590445 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:15.590240 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" event={"ID":"98bb3a5f-ee94-4775-8f3d-affb7244c0cc","Type":"ContainerStarted","Data":"e8f4e371953b79d1f85525d7c2d1d301e9214350d4ce69fd70791f8d4434cf71"} Jul 12 15:49:15.606827 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:15.606786 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" podStartSLOduration=1.138519407 podStartE2EDuration="1.60677278s" totalImagesPullingTime="468.253373ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:49:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:49:15.605120696 +0000 UTC m=+1733.313928290" watchObservedRunningTime="2026-07-12 15:49:15.60677278 +0000 UTC m=+1733.315580374" Jul 12 15:49:19.484389 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.484352 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:19.484762 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.484531 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" podUID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" containerName="maas-api" containerID="cri-o://ff2a00ac50cfeaef6d412000a790d29772ea3767e9b6d8e74b80c5a748605811" gracePeriod=30 Jul 12 15:49:19.609804 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.609768 2567 generic.go:356] "Generic (PLEG): container finished" podID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" containerID="ff2a00ac50cfeaef6d412000a790d29772ea3767e9b6d8e74b80c5a748605811" exitCode=0 Jul 12 15:49:19.609941 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.609848 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" event={"ID":"98bb3a5f-ee94-4775-8f3d-affb7244c0cc","Type":"ContainerDied","Data":"ff2a00ac50cfeaef6d412000a790d29772ea3767e9b6d8e74b80c5a748605811"} Jul 12 15:49:19.723707 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.723681 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:19.732100 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.732074 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca\") pod \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " Jul 12 15:49:19.732199 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.732128 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7xb85\" (UniqueName: \"kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85\") pod \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " Jul 12 15:49:19.732240 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.732220 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls\") pod \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\" (UID: \"98bb3a5f-ee94-4775-8f3d-affb7244c0cc\") " Jul 12 15:49:19.732454 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.732418 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "98bb3a5f-ee94-4775-8f3d-affb7244c0cc" (UID: "98bb3a5f-ee94-4775-8f3d-affb7244c0cc"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:49:19.732520 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.732496 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:19.734299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.734275 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85" (OuterVolumeSpecName: "kube-api-access-7xb85") pod "98bb3a5f-ee94-4775-8f3d-affb7244c0cc" (UID: "98bb3a5f-ee94-4775-8f3d-affb7244c0cc"). InnerVolumeSpecName "kube-api-access-7xb85". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:49:19.734299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.734293 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "98bb3a5f-ee94-4775-8f3d-affb7244c0cc" (UID: "98bb3a5f-ee94-4775-8f3d-affb7244c0cc"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:49:19.832836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.832799 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7xb85\" (UniqueName: \"kubernetes.io/projected/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-kube-api-access-7xb85\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:19.832836 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:19.832829 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/98bb3a5f-ee94-4775-8f3d-affb7244c0cc-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:49:20.616174 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.616137 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" event={"ID":"98bb3a5f-ee94-4775-8f3d-affb7244c0cc","Type":"ContainerDied","Data":"e8f4e371953b79d1f85525d7c2d1d301e9214350d4ce69fd70791f8d4434cf71"} Jul 12 15:49:20.616612 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.616182 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f" Jul 12 15:49:20.616612 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.616204 2567 scope.go:117] "RemoveContainer" containerID="ff2a00ac50cfeaef6d412000a790d29772ea3767e9b6d8e74b80c5a748605811" Jul 12 15:49:20.636967 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.636940 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:20.638919 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.638885 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-5d5046a9-7fff4cdf5c-fsp4f"] Jul 12 15:49:20.846092 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:49:20.846034 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" path="/var/lib/kubelet/pods/98bb3a5f-ee94-4775-8f3d-affb7244c0cc/volumes" Jul 12 15:50:22.856023 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:50:22.855989 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:50:22.862832 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:50:22.862810 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:50:22.863425 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:50:22.863408 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:50:22.870131 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:50:22.870114 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:51:20.681859 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.681820 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:20.682467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.682444 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" containerName="maas-api" Jul 12 15:51:20.682467 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.682461 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" containerName="maas-api" Jul 12 15:51:20.682582 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.682568 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="98bb3a5f-ee94-4775-8f3d-affb7244c0cc" containerName="maas-api" Jul 12 15:51:20.686090 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.686069 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.694069 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.688769 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-4d006861-serving-cert\"" Jul 12 15:51:20.694069 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.688791 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:51:20.700362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.700339 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:20.797944 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.797907 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.798153 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.797982 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.798153 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.798022 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-482fh\" (UniqueName: \"kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.899011 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.898977 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.899199 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.899032 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.899199 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.899083 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-482fh\" (UniqueName: \"kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.899609 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.899588 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.901532 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.901513 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:20.907299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.907279 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-482fh\" (UniqueName: \"kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh\") pod \"maas-api-e2e-mt-4d006861-7464bfc67-nd9df\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:21.000067 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:20.999959 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:21.133032 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:21.132997 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:21.136425 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:51:21.136399 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7fe72d61_a212_4ae1_83a3_e3fd876acf76.slice/crio-46fc2072ee01f3993923fe26f3202a53bfc928a1b400b0eebd251e7023d40aa4 WatchSource:0}: Error finding container 46fc2072ee01f3993923fe26f3202a53bfc928a1b400b0eebd251e7023d40aa4: Status 404 returned error can't find the container with id 46fc2072ee01f3993923fe26f3202a53bfc928a1b400b0eebd251e7023d40aa4 Jul 12 15:51:21.162382 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:21.162350 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" event={"ID":"7fe72d61-a212-4ae1-83a3-e3fd876acf76","Type":"ContainerStarted","Data":"46fc2072ee01f3993923fe26f3202a53bfc928a1b400b0eebd251e7023d40aa4"} Jul 12 15:51:22.168773 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:22.168733 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" event={"ID":"7fe72d61-a212-4ae1-83a3-e3fd876acf76","Type":"ContainerStarted","Data":"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6"} Jul 12 15:51:22.169156 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:22.168853 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:22.186221 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:22.186179 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" podStartSLOduration=1.648825226 podStartE2EDuration="2.186164787s" totalImagesPullingTime="537.339561ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:51:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:51:22.185342453 +0000 UTC m=+1859.894150049" watchObservedRunningTime="2026-07-12 15:51:22.186164787 +0000 UTC m=+1859.894972381" Jul 12 15:51:28.179374 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:28.179345 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:32.263331 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.263298 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:32.263789 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.263558 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" podUID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" containerName="maas-api" containerID="cri-o://baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6" gracePeriod=30 Jul 12 15:51:32.508487 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.508463 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:32.600691 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.600662 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-482fh\" (UniqueName: \"kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh\") pod \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " Jul 12 15:51:32.600865 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.600732 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls\") pod \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " Jul 12 15:51:32.600865 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.600796 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca\") pod \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\" (UID: \"7fe72d61-a212-4ae1-83a3-e3fd876acf76\") " Jul 12 15:51:32.601204 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.601168 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7fe72d61-a212-4ae1-83a3-e3fd876acf76" (UID: "7fe72d61-a212-4ae1-83a3-e3fd876acf76"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:51:32.602916 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.602892 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7fe72d61-a212-4ae1-83a3-e3fd876acf76" (UID: "7fe72d61-a212-4ae1-83a3-e3fd876acf76"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:51:32.602916 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.602906 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh" (OuterVolumeSpecName: "kube-api-access-482fh") pod "7fe72d61-a212-4ae1-83a3-e3fd876acf76" (UID: "7fe72d61-a212-4ae1-83a3-e3fd876acf76"). InnerVolumeSpecName "kube-api-access-482fh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:51:32.701959 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.701931 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7fe72d61-a212-4ae1-83a3-e3fd876acf76-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:51:32.701959 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.701956 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7fe72d61-a212-4ae1-83a3-e3fd876acf76-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:51:32.702168 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:32.701966 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-482fh\" (UniqueName: \"kubernetes.io/projected/7fe72d61-a212-4ae1-83a3-e3fd876acf76-kube-api-access-482fh\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:51:33.221675 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.221581 2567 generic.go:356] "Generic (PLEG): container finished" podID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" containerID="baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6" exitCode=0 Jul 12 15:51:33.221675 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.221625 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" Jul 12 15:51:33.221906 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.221681 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" event={"ID":"7fe72d61-a212-4ae1-83a3-e3fd876acf76","Type":"ContainerDied","Data":"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6"} Jul 12 15:51:33.221906 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.221736 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df" event={"ID":"7fe72d61-a212-4ae1-83a3-e3fd876acf76","Type":"ContainerDied","Data":"46fc2072ee01f3993923fe26f3202a53bfc928a1b400b0eebd251e7023d40aa4"} Jul 12 15:51:33.221906 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.221768 2567 scope.go:117] "RemoveContainer" containerID="baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6" Jul 12 15:51:33.230509 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.230494 2567 scope.go:117] "RemoveContainer" containerID="baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6" Jul 12 15:51:33.230780 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:51:33.230762 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6\": container with ID starting with baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6 not found: ID does not exist" containerID="baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6" Jul 12 15:51:33.230866 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.230788 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6"} err="failed to get container status \"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6\": rpc error: code = NotFound desc = could not find container \"baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6\": container with ID starting with baee7acd537aa423862cf86b990f12c79dd45521c32aa18ce4c8f6a3dea8c4a6 not found: ID does not exist" Jul 12 15:51:33.237373 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.237346 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:33.240548 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:33.240528 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-4d006861-7464bfc67-nd9df"] Jul 12 15:51:34.846110 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:51:34.846080 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" path="/var/lib/kubelet/pods/7fe72d61-a212-4ae1-83a3-e3fd876acf76/volumes" Jul 12 15:52:00.590304 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.590215 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:00.590861 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.590775 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" containerName="maas-api" Jul 12 15:52:00.590861 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.590796 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" containerName="maas-api" Jul 12 15:52:00.590986 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.590908 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="7fe72d61-a212-4ae1-83a3-e3fd876acf76" containerName="maas-api" Jul 12 15:52:00.594041 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.594018 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.596989 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.596792 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-cd9517a2-serving-cert\"" Jul 12 15:52:00.596989 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.596831 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:52:00.603256 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.603234 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:00.746219 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.746188 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.746389 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.746226 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.746389 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.746256 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22g7h\" (UniqueName: \"kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.847668 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.847573 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.847668 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.847633 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.847668 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.847654 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22g7h\" (UniqueName: \"kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.848224 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.848205 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.850151 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.850132 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.856329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.856299 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22g7h\" (UniqueName: \"kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h\") pod \"maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:00.906752 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:00.906728 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:01.047734 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:01.047545 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:01.050572 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:52:01.050541 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c44024a_181e_412a_a13c_667f16c63f8d.slice/crio-5834dcf76351636ada2a24ff2d8ddcf367ecdfca201cae50b5f1e05bf7a1cdd9 WatchSource:0}: Error finding container 5834dcf76351636ada2a24ff2d8ddcf367ecdfca201cae50b5f1e05bf7a1cdd9: Status 404 returned error can't find the container with id 5834dcf76351636ada2a24ff2d8ddcf367ecdfca201cae50b5f1e05bf7a1cdd9 Jul 12 15:52:01.351031 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:01.351001 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" event={"ID":"2c44024a-181e-412a-a13c-667f16c63f8d","Type":"ContainerStarted","Data":"5834dcf76351636ada2a24ff2d8ddcf367ecdfca201cae50b5f1e05bf7a1cdd9"} Jul 12 15:52:02.357294 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:02.357256 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" event={"ID":"2c44024a-181e-412a-a13c-667f16c63f8d","Type":"ContainerStarted","Data":"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e"} Jul 12 15:52:02.357689 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:02.357387 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:02.376172 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:02.376124 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" podStartSLOduration=2.059023174 podStartE2EDuration="2.376107945s" totalImagesPullingTime="317.084771ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:52:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:52:02.373790025 +0000 UTC m=+1900.082597633" watchObservedRunningTime="2026-07-12 15:52:02.376107945 +0000 UTC m=+1900.084915538" Jul 12 15:52:08.367362 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:08.367331 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:32.888568 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:32.888531 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:32.889201 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:32.888988 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" podUID="2c44024a-181e-412a-a13c-667f16c63f8d" containerName="maas-api" containerID="cri-o://8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e" gracePeriod=30 Jul 12 15:52:33.144962 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.144892 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:33.237240 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.237208 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca\") pod \"2c44024a-181e-412a-a13c-667f16c63f8d\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " Jul 12 15:52:33.237240 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.237243 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls\") pod \"2c44024a-181e-412a-a13c-667f16c63f8d\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " Jul 12 15:52:33.237438 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.237300 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-22g7h\" (UniqueName: \"kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h\") pod \"2c44024a-181e-412a-a13c-667f16c63f8d\" (UID: \"2c44024a-181e-412a-a13c-667f16c63f8d\") " Jul 12 15:52:33.237601 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.237567 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2c44024a-181e-412a-a13c-667f16c63f8d" (UID: "2c44024a-181e-412a-a13c-667f16c63f8d"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:52:33.239460 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.239436 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h" (OuterVolumeSpecName: "kube-api-access-22g7h") pod "2c44024a-181e-412a-a13c-667f16c63f8d" (UID: "2c44024a-181e-412a-a13c-667f16c63f8d"). InnerVolumeSpecName "kube-api-access-22g7h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:52:33.239533 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.239437 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2c44024a-181e-412a-a13c-667f16c63f8d" (UID: "2c44024a-181e-412a-a13c-667f16c63f8d"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:52:33.338708 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.338679 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c44024a-181e-412a-a13c-667f16c63f8d-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:52:33.338708 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.338705 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c44024a-181e-412a-a13c-667f16c63f8d-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:52:33.338708 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.338715 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-22g7h\" (UniqueName: \"kubernetes.io/projected/2c44024a-181e-412a-a13c-667f16c63f8d-kube-api-access-22g7h\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:52:33.508188 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.508102 2567 generic.go:356] "Generic (PLEG): container finished" podID="2c44024a-181e-412a-a13c-667f16c63f8d" containerID="8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e" exitCode=0 Jul 12 15:52:33.508188 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.508156 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" Jul 12 15:52:33.508188 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.508164 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" event={"ID":"2c44024a-181e-412a-a13c-667f16c63f8d","Type":"ContainerDied","Data":"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e"} Jul 12 15:52:33.508443 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.508214 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67" event={"ID":"2c44024a-181e-412a-a13c-667f16c63f8d","Type":"ContainerDied","Data":"5834dcf76351636ada2a24ff2d8ddcf367ecdfca201cae50b5f1e05bf7a1cdd9"} Jul 12 15:52:33.508443 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.508239 2567 scope.go:117] "RemoveContainer" containerID="8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e" Jul 12 15:52:33.517314 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.517291 2567 scope.go:117] "RemoveContainer" containerID="8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e" Jul 12 15:52:33.517565 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:52:33.517541 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e\": container with ID starting with 8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e not found: ID does not exist" containerID="8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e" Jul 12 15:52:33.517674 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.517574 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e"} err="failed to get container status \"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e\": rpc error: code = NotFound desc = could not find container \"8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e\": container with ID starting with 8f87e34848faf9656b70b442a4c62fda96982450cd461663c9fe02d6c583878e not found: ID does not exist" Jul 12 15:52:33.531661 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.531640 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:33.535135 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:33.535113 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-cd9517a2-7b46bb6b9b-bgv67"] Jul 12 15:52:34.846068 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:52:34.846007 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c44024a-181e-412a-a13c-667f16c63f8d" path="/var/lib/kubelet/pods/2c44024a-181e-412a-a13c-667f16c63f8d/volumes" Jul 12 15:54:36.249470 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.249439 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:54:36.249948 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.249796 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c44024a-181e-412a-a13c-667f16c63f8d" containerName="maas-api" Jul 12 15:54:36.249948 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.249807 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c44024a-181e-412a-a13c-667f16c63f8d" containerName="maas-api" Jul 12 15:54:36.249948 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.249874 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c44024a-181e-412a-a13c-667f16c63f8d" containerName="maas-api" Jul 12 15:54:36.252957 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.252914 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.255903 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.255875 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-0e354c-serving-cert\"" Jul 12 15:54:36.256020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.255885 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:54:36.266243 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.266219 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:54:36.398710 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.398673 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.398899 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.398723 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.398899 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.398805 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6l7qw\" (UniqueName: \"kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.499381 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.499350 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.499621 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.499411 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.499621 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.499456 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6l7qw\" (UniqueName: \"kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.500168 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.500147 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.502017 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.501998 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.508166 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.508133 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6l7qw\" (UniqueName: \"kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw\") pod \"maas-api-e2e-inf-b-0e354c-db67c689-5v9fk\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.563808 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.563777 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:36.694952 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.694929 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:54:36.697715 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:54:36.697686 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5992036c_7b11_4c57_8f41_cc18d3958aff.slice/crio-b08e0ed7e2e1d8b18a145fb9d4d3996a3b4e7cc7b0f3e2085307e9e502e95f0a WatchSource:0}: Error finding container b08e0ed7e2e1d8b18a145fb9d4d3996a3b4e7cc7b0f3e2085307e9e502e95f0a: Status 404 returned error can't find the container with id b08e0ed7e2e1d8b18a145fb9d4d3996a3b4e7cc7b0f3e2085307e9e502e95f0a Jul 12 15:54:36.699176 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:36.699157 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 15:54:37.070879 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:37.070853 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" event={"ID":"5992036c-7b11-4c57-8f41-cc18d3958aff","Type":"ContainerStarted","Data":"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362"} Jul 12 15:54:37.071025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:37.070911 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" event={"ID":"5992036c-7b11-4c57-8f41-cc18d3958aff","Type":"ContainerStarted","Data":"b08e0ed7e2e1d8b18a145fb9d4d3996a3b4e7cc7b0f3e2085307e9e502e95f0a"} Jul 12 15:54:37.071025 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:37.070987 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:54:37.089245 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:37.089204 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" podStartSLOduration=0.792417615 podStartE2EDuration="1.089189127s" totalImagesPullingTime="296.771512ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:54:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:54:37.087526838 +0000 UTC m=+2054.796334454" watchObservedRunningTime="2026-07-12 15:54:37.089189127 +0000 UTC m=+2054.797996721" Jul 12 15:54:43.081074 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:54:43.080973 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:55:05.200849 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.200817 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:05.204419 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.204398 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.207300 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.207275 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0e354c\"/\"openshift-service-ca.crt\"" Jul 12 15:55:05.208487 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.208465 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0e354c\"/\"default-dockercfg-vpv2h\"" Jul 12 15:55:05.208613 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.208488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-0e354c\"/\"test-model-0e354c-kserve-self-signed-certs\"" Jul 12 15:55:05.208613 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.208515 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-0e354c\"/\"kube-root-ca.crt\"" Jul 12 15:55:05.217193 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.217170 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:05.351250 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.351250 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351249 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.351525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351274 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.351525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351427 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m5b56\" (UniqueName: \"kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.351525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351475 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.351664 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.351549 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.453571 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.453480 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.453841 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.453818 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.453953 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.453939 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.454070 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.454036 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.454238 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.454222 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m5b56\" (UniqueName: \"kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.454355 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.454343 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.454509 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.454384 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.454597 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.454120 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.455250 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.455227 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.459366 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.459322 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.459857 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.459832 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.462317 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.462295 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m5b56\" (UniqueName: \"kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56\") pod \"test-model-0e354c-kserve-8ddd4f7fc-s8fh9\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.515180 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.515151 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:05.649230 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:05.649136 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:05.651535 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:55:05.651504 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03349b7_f1cc_4bb3_8ff7_3952801184a2.slice/crio-d73555c3403cb30f3bf26a5bc3d876036a285529a3a5a924c9e2cc02102682db WatchSource:0}: Error finding container d73555c3403cb30f3bf26a5bc3d876036a285529a3a5a924c9e2cc02102682db: Status 404 returned error can't find the container with id d73555c3403cb30f3bf26a5bc3d876036a285529a3a5a924c9e2cc02102682db Jul 12 15:55:06.210792 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:06.210753 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" event={"ID":"e03349b7-f1cc-4bb3-8ff7-3952801184a2","Type":"ContainerStarted","Data":"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f"} Jul 12 15:55:06.211170 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:06.210800 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" event={"ID":"e03349b7-f1cc-4bb3-8ff7-3952801184a2","Type":"ContainerStarted","Data":"d73555c3403cb30f3bf26a5bc3d876036a285529a3a5a924c9e2cc02102682db"} Jul 12 15:55:06.231071 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:06.230218 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" podStartSLOduration=1.230193886 podStartE2EDuration="1.230193886s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:55:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:55:06.227959278 +0000 UTC m=+2083.936766873" watchObservedRunningTime="2026-07-12 15:55:06.230193886 +0000 UTC m=+2083.939001481" Jul 12 15:55:15.515623 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:15.515584 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:15.515623 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:15.515632 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:15.527955 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:15.527930 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:16.267810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:16.267782 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:22.884674 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:22.884646 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:55:22.892031 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:22.892011 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:55:22.893392 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:22.893373 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 15:55:22.900185 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:22.900166 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 15:55:39.997399 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:39.997366 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:55.019017 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.018957 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" podUID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" containerName="main" containerID="cri-o://847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f" gracePeriod=45 Jul 12 15:55:55.261833 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.261809 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:55.384087 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384035 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384087 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384098 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m5b56\" (UniqueName: \"kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384367 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384123 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384367 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384140 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384367 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384187 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384367 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384206 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs\") pod \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\" (UID: \"e03349b7-f1cc-4bb3-8ff7-3952801184a2\") " Jul 12 15:55:55.384567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384462 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:55:55.384567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384452 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home" (OuterVolumeSpecName: "home") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:55:55.384567 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.384478 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache" (OuterVolumeSpecName: "model-cache") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:55:55.386569 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.386546 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm" (OuterVolumeSpecName: "dshm") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 15:55:55.386569 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.386555 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:55:55.386702 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.386572 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56" (OuterVolumeSpecName: "kube-api-access-m5b56") pod "e03349b7-f1cc-4bb3-8ff7-3952801184a2" (UID: "e03349b7-f1cc-4bb3-8ff7-3952801184a2"). InnerVolumeSpecName "kube-api-access-m5b56". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:55:55.441383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.441355 2567 generic.go:356] "Generic (PLEG): container finished" podID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" containerID="847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f" exitCode=0 Jul 12 15:55:55.441525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.441397 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" Jul 12 15:55:55.441525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.441453 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" event={"ID":"e03349b7-f1cc-4bb3-8ff7-3952801184a2","Type":"ContainerDied","Data":"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f"} Jul 12 15:55:55.441525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.441493 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9" event={"ID":"e03349b7-f1cc-4bb3-8ff7-3952801184a2","Type":"ContainerDied","Data":"d73555c3403cb30f3bf26a5bc3d876036a285529a3a5a924c9e2cc02102682db"} Jul 12 15:55:55.441525 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.441511 2567 scope.go:117] "RemoveContainer" containerID="847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f" Jul 12 15:55:55.450533 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.450516 2567 scope.go:117] "RemoveContainer" containerID="847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f" Jul 12 15:55:55.450800 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:55:55.450782 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f\": container with ID starting with 847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f not found: ID does not exist" containerID="847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f" Jul 12 15:55:55.450867 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.450805 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f"} err="failed to get container status \"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f\": rpc error: code = NotFound desc = could not find container \"847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f\": container with ID starting with 847e9a7115e514a4f5be26e55eb2881e7f3ca9324ec17cb841ffffe657b81b7f not found: ID does not exist" Jul 12 15:55:55.461398 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.461376 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:55.464933 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.464912 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-0e354c/test-model-0e354c-kserve-8ddd4f7fc-s8fh9"] Jul 12 15:55:55.485178 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485158 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-dshm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:55.485178 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485178 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m5b56\" (UniqueName: \"kubernetes.io/projected/e03349b7-f1cc-4bb3-8ff7-3952801184a2-kube-api-access-m5b56\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:55.485299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485188 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tmp-dir\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:55.485299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485198 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-model-cache\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:55.485299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485206 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03349b7-f1cc-4bb3-8ff7-3952801184a2-home\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:55.485299 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:55.485214 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03349b7-f1cc-4bb3-8ff7-3952801184a2-tls-certs\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:55:56.845787 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:55:56.845751 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" path="/var/lib/kubelet/pods/e03349b7-f1cc-4bb3-8ff7-3952801184a2/volumes" Jul 12 15:56:23.759247 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:23.759200 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:56:23.759688 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:23.759462 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" podUID="5992036c-7b11-4c57-8f41-cc18d3958aff" containerName="maas-api" containerID="cri-o://56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362" gracePeriod=30 Jul 12 15:56:25.005417 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.005391 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:56:25.031230 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.031206 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca\") pod \"5992036c-7b11-4c57-8f41-cc18d3958aff\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " Jul 12 15:56:25.031353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.031254 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls\") pod \"5992036c-7b11-4c57-8f41-cc18d3958aff\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " Jul 12 15:56:25.031353 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.031281 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6l7qw\" (UniqueName: \"kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw\") pod \"5992036c-7b11-4c57-8f41-cc18d3958aff\" (UID: \"5992036c-7b11-4c57-8f41-cc18d3958aff\") " Jul 12 15:56:25.031512 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.031489 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "5992036c-7b11-4c57-8f41-cc18d3958aff" (UID: "5992036c-7b11-4c57-8f41-cc18d3958aff"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:56:25.033454 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.033431 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw" (OuterVolumeSpecName: "kube-api-access-6l7qw") pod "5992036c-7b11-4c57-8f41-cc18d3958aff" (UID: "5992036c-7b11-4c57-8f41-cc18d3958aff"). InnerVolumeSpecName "kube-api-access-6l7qw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:25.033550 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.033465 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "5992036c-7b11-4c57-8f41-cc18d3958aff" (UID: "5992036c-7b11-4c57-8f41-cc18d3958aff"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:56:25.132807 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.132732 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/5992036c-7b11-4c57-8f41-cc18d3958aff-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:25.132807 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.132759 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/5992036c-7b11-4c57-8f41-cc18d3958aff-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:25.132807 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.132770 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6l7qw\" (UniqueName: \"kubernetes.io/projected/5992036c-7b11-4c57-8f41-cc18d3958aff-kube-api-access-6l7qw\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:25.584749 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.584712 2567 generic.go:356] "Generic (PLEG): container finished" podID="5992036c-7b11-4c57-8f41-cc18d3958aff" containerID="56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362" exitCode=0 Jul 12 15:56:25.584908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.584761 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" Jul 12 15:56:25.584908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.584769 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" event={"ID":"5992036c-7b11-4c57-8f41-cc18d3958aff","Type":"ContainerDied","Data":"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362"} Jul 12 15:56:25.584908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.584821 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk" event={"ID":"5992036c-7b11-4c57-8f41-cc18d3958aff","Type":"ContainerDied","Data":"b08e0ed7e2e1d8b18a145fb9d4d3996a3b4e7cc7b0f3e2085307e9e502e95f0a"} Jul 12 15:56:25.584908 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.584851 2567 scope.go:117] "RemoveContainer" containerID="56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362" Jul 12 15:56:25.595512 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.595490 2567 scope.go:117] "RemoveContainer" containerID="56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362" Jul 12 15:56:25.595792 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:56:25.595768 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362\": container with ID starting with 56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362 not found: ID does not exist" containerID="56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362" Jul 12 15:56:25.595872 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.595801 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362"} err="failed to get container status \"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362\": rpc error: code = NotFound desc = could not find container \"56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362\": container with ID starting with 56e68b67245263d5558393eeaa12636ebb4c1b7fb6b58fa9f0e85593f7fc6362 not found: ID does not exist" Jul 12 15:56:25.605721 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.605695 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:56:25.609108 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:25.609087 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-0e354c-db67c689-5v9fk"] Jul 12 15:56:26.846713 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:26.846673 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5992036c-7b11-4c57-8f41-cc18d3958aff" path="/var/lib/kubelet/pods/5992036c-7b11-4c57-8f41-cc18d3958aff/volumes" Jul 12 15:56:33.871810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.871775 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:33.872283 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872134 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" containerName="main" Jul 12 15:56:33.872283 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872146 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" containerName="main" Jul 12 15:56:33.872283 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872166 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5992036c-7b11-4c57-8f41-cc18d3958aff" containerName="maas-api" Jul 12 15:56:33.872283 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872176 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="5992036c-7b11-4c57-8f41-cc18d3958aff" containerName="maas-api" Jul 12 15:56:33.872429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872290 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03349b7-f1cc-4bb3-8ff7-3952801184a2" containerName="main" Jul 12 15:56:33.872429 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.872303 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="5992036c-7b11-4c57-8f41-cc18d3958aff" containerName="maas-api" Jul 12 15:56:33.876832 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.876814 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:33.880507 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.880488 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-a5af97-serving-cert\"" Jul 12 15:56:33.880606 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.880531 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:56:33.898679 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:33.898640 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:34.011111 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.011078 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.011267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.011122 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dgx7\" (UniqueName: \"kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.011267 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.011213 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.112069 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.112015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.112236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.112116 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dgx7\" (UniqueName: \"kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.112236 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.112193 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.112829 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.112805 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.114612 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.114591 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.120548 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.120519 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dgx7\" (UniqueName: \"kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7\") pod \"maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.186712 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.186650 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:34.322361 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.322334 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:34.324942 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:56:34.324913 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84d5d760_58dd_433e_8894_40743c059556.slice/crio-524a792308631d2a4d17416c64561874c52fbbb0dd6f167b031c78b15e8a122a WatchSource:0}: Error finding container 524a792308631d2a4d17416c64561874c52fbbb0dd6f167b031c78b15e8a122a: Status 404 returned error can't find the container with id 524a792308631d2a4d17416c64561874c52fbbb0dd6f167b031c78b15e8a122a Jul 12 15:56:34.633764 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:34.633735 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" event={"ID":"84d5d760-58dd-433e-8894-40743c059556","Type":"ContainerStarted","Data":"524a792308631d2a4d17416c64561874c52fbbb0dd6f167b031c78b15e8a122a"} Jul 12 15:56:35.639434 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:35.639401 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" event={"ID":"84d5d760-58dd-433e-8894-40743c059556","Type":"ContainerStarted","Data":"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1"} Jul 12 15:56:35.639839 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:35.639464 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:35.656594 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:35.656542 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" podStartSLOduration=2.265634935 podStartE2EDuration="2.656526438s" totalImagesPullingTime="390.891503ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:56:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:56:35.655209298 +0000 UTC m=+2173.364016906" watchObservedRunningTime="2026-07-12 15:56:35.656526438 +0000 UTC m=+2173.365334048" Jul 12 15:56:41.649329 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:41.649300 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:54.856699 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:54.856668 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:54.857258 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:54.856850 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" podUID="84d5d760-58dd-433e-8894-40743c059556" containerName="maas-api" containerID="cri-o://04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1" gracePeriod=30 Jul 12 15:56:55.108898 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.108845 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:55.191795 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.191764 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca\") pod \"84d5d760-58dd-433e-8894-40743c059556\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " Jul 12 15:56:55.191942 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.191819 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dgx7\" (UniqueName: \"kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7\") pod \"84d5d760-58dd-433e-8894-40743c059556\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " Jul 12 15:56:55.191942 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.191880 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls\") pod \"84d5d760-58dd-433e-8894-40743c059556\" (UID: \"84d5d760-58dd-433e-8894-40743c059556\") " Jul 12 15:56:55.192207 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.192179 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "84d5d760-58dd-433e-8894-40743c059556" (UID: "84d5d760-58dd-433e-8894-40743c059556"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 15:56:55.194020 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.193990 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7" (OuterVolumeSpecName: "kube-api-access-4dgx7") pod "84d5d760-58dd-433e-8894-40743c059556" (UID: "84d5d760-58dd-433e-8894-40743c059556"). InnerVolumeSpecName "kube-api-access-4dgx7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 15:56:55.194228 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.194135 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "84d5d760-58dd-433e-8894-40743c059556" (UID: "84d5d760-58dd-433e-8894-40743c059556"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 15:56:55.292723 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.292690 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/84d5d760-58dd-433e-8894-40743c059556-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:55.292723 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.292718 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dgx7\" (UniqueName: \"kubernetes.io/projected/84d5d760-58dd-433e-8894-40743c059556-kube-api-access-4dgx7\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:55.292869 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.292733 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/84d5d760-58dd-433e-8894-40743c059556-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 15:56:55.734777 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.734740 2567 generic.go:356] "Generic (PLEG): container finished" podID="84d5d760-58dd-433e-8894-40743c059556" containerID="04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1" exitCode=0 Jul 12 15:56:55.734934 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.734789 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" Jul 12 15:56:55.734934 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.734792 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" event={"ID":"84d5d760-58dd-433e-8894-40743c059556","Type":"ContainerDied","Data":"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1"} Jul 12 15:56:55.734934 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.734838 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6" event={"ID":"84d5d760-58dd-433e-8894-40743c059556","Type":"ContainerDied","Data":"524a792308631d2a4d17416c64561874c52fbbb0dd6f167b031c78b15e8a122a"} Jul 12 15:56:55.734934 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.734864 2567 scope.go:117] "RemoveContainer" containerID="04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1" Jul 12 15:56:55.743833 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.743814 2567 scope.go:117] "RemoveContainer" containerID="04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1" Jul 12 15:56:55.744112 ip-10-0-136-30 kubenswrapper[2567]: E0712 15:56:55.744096 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1\": container with ID starting with 04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1 not found: ID does not exist" containerID="04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1" Jul 12 15:56:55.744170 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.744120 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1"} err="failed to get container status \"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1\": rpc error: code = NotFound desc = could not find container \"04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1\": container with ID starting with 04f70b33c054bb271d5437b3cfad764753498362f200b06af3c4eea84073f9f1 not found: ID does not exist" Jul 12 15:56:55.755407 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.755386 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:55.759018 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:55.758991 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-a5af97-f759cf7f8-frcq6"] Jul 12 15:56:56.845952 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:56:56.845917 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84d5d760-58dd-433e-8894-40743c059556" path="/var/lib/kubelet/pods/84d5d760-58dd-433e-8894-40743c059556/volumes" Jul 12 15:57:19.188282 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.188241 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 15:57:19.188810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.188791 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84d5d760-58dd-433e-8894-40743c059556" containerName="maas-api" Jul 12 15:57:19.188810 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.188811 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="84d5d760-58dd-433e-8894-40743c059556" containerName="maas-api" Jul 12 15:57:19.188952 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.188892 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="84d5d760-58dd-433e-8894-40743c059556" containerName="maas-api" Jul 12 15:57:19.193915 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.193889 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.196813 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.196789 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-3da434-serving-cert\"" Jul 12 15:57:19.196958 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.196844 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-mc8ms\"" Jul 12 15:57:19.200901 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.200878 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 15:57:19.295572 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.295545 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.295724 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.295575 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wk9f8\" (UniqueName: \"kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.295724 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.295616 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.396565 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.396530 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.396565 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.396560 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wk9f8\" (UniqueName: \"kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.396773 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.396596 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.397295 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.397275 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.399095 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.399064 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.405664 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.405636 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wk9f8\" (UniqueName: \"kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8\") pod \"maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.507991 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.507916 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:19.639344 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.639290 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 15:57:19.641716 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:57:19.641689 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4c80bc6_d145_47bf_86e9_468491e9d872.slice/crio-97a36494fe5fe23072b96236943119b089e4cba19d74f19fb6e3873c054e6d2d WatchSource:0}: Error finding container 97a36494fe5fe23072b96236943119b089e4cba19d74f19fb6e3873c054e6d2d: Status 404 returned error can't find the container with id 97a36494fe5fe23072b96236943119b089e4cba19d74f19fb6e3873c054e6d2d Jul 12 15:57:19.848106 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:19.848072 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" event={"ID":"b4c80bc6-d145-47bf-86e9-468491e9d872","Type":"ContainerStarted","Data":"97a36494fe5fe23072b96236943119b089e4cba19d74f19fb6e3873c054e6d2d"} Jul 12 15:57:20.854837 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:20.854804 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" event={"ID":"b4c80bc6-d145-47bf-86e9-468491e9d872","Type":"ContainerStarted","Data":"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba"} Jul 12 15:57:20.855241 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:20.854898 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:20.872952 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:20.872909 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" podStartSLOduration=1.609395739 podStartE2EDuration="1.872895277s" totalImagesPullingTime="263.499538ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:57:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:57:20.871401939 +0000 UTC m=+2218.580209535" watchObservedRunningTime="2026-07-12 15:57:20.872895277 +0000 UTC m=+2218.581702871" Jul 12 15:57:26.864648 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:26.864618 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 15:57:44.793111 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.793008 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 15:57:44.796249 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.796233 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.798924 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.798903 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a99179\"/\"kube-root-ca.crt\"" Jul 12 15:57:44.800276 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.800245 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a99179\"/\"auth-test-model-a99179-kserve-self-signed-certs\"" Jul 12 15:57:44.800397 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.800288 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a99179\"/\"openshift-service-ca.crt\"" Jul 12 15:57:44.800397 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.800294 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a99179\"/\"default-dockercfg-9jwlz\"" Jul 12 15:57:44.806273 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.806253 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 15:57:44.909910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.909877 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.909910 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.909912 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rwbpm\" (UniqueName: \"kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.910119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.909961 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.910119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.909995 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.910119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.910084 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:44.910119 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:44.910109 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011226 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011200 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011379 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011242 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011379 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011273 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011379 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011347 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011511 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011375 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rwbpm\" (UniqueName: \"kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011511 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011430 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011717 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011693 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011795 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011701 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.011795 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.011770 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.013587 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.013561 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.013913 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.013896 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.019489 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.019465 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rwbpm\" (UniqueName: \"kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm\") pod \"auth-test-model-a99179-kserve-74c9c54775-7mplg\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.108387 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.108353 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:45.241179 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.241155 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 15:57:45.243221 ip-10-0-136-30 kubenswrapper[2567]: W0712 15:57:45.243191 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode03ae7e2_8c96_4c11_9be5_ad3c99b78aa2.slice/crio-8c0bac32f6be62a2873ba33781c7457219135b0f314fcd61bf6bb5b5fc9e2e4e WatchSource:0}: Error finding container 8c0bac32f6be62a2873ba33781c7457219135b0f314fcd61bf6bb5b5fc9e2e4e: Status 404 returned error can't find the container with id 8c0bac32f6be62a2873ba33781c7457219135b0f314fcd61bf6bb5b5fc9e2e4e Jul 12 15:57:45.968112 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.968075 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" event={"ID":"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2","Type":"ContainerStarted","Data":"96d8f2fc60ce791f5ec1d977841e9ec12fd2af724beb166c028cd03d804fc10d"} Jul 12 15:57:45.968516 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.968120 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" event={"ID":"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2","Type":"ContainerStarted","Data":"8c0bac32f6be62a2873ba33781c7457219135b0f314fcd61bf6bb5b5fc9e2e4e"} Jul 12 15:57:45.983196 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:45.983145 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" podStartSLOduration=1.983130467 podStartE2EDuration="1.983130467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 15:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 15:57:45.982819562 +0000 UTC m=+2243.691627176" watchObservedRunningTime="2026-07-12 15:57:45.983130467 +0000 UTC m=+2243.691938062" Jul 12 15:57:55.109088 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:55.109037 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:55.109088 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:55.109095 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:55.121468 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:55.121445 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 15:57:56.023383 ip-10-0-136-30 kubenswrapper[2567]: I0712 15:57:56.023355 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 16:00:00.135139 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.135106 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp"] Jul 12 16:00:00.138554 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.138536 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:00.148324 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.148291 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp"] Jul 12 16:00:00.263622 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.263577 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wphv9\" (UniqueName: \"kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9\") pod \"maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp\" (UID: \"27d95005-ac01-42e6-a885-0861221d23cc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:00.365063 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.365015 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wphv9\" (UniqueName: \"kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9\") pod \"maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp\" (UID: \"27d95005-ac01-42e6-a885-0861221d23cc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:00.374779 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.374751 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wphv9\" (UniqueName: \"kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9\") pod \"maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp\" (UID: \"27d95005-ac01-42e6-a885-0861221d23cc\") " pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:00.451530 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.451433 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:00.585516 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.585489 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp"] Jul 12 16:00:00.588108 ip-10-0-136-30 kubenswrapper[2567]: W0712 16:00:00.588077 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27d95005_ac01_42e6_a885_0861221d23cc.slice/crio-a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8 WatchSource:0}: Error finding container a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8: Status 404 returned error can't find the container with id a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8 Jul 12 16:00:00.590112 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.590089 2567 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 16:00:00.600881 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:00.600855 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" event={"ID":"27d95005-ac01-42e6-a885-0861221d23cc","Type":"ContainerStarted","Data":"a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8"} Jul 12 16:00:01.553931 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:00:01.553891 2567 log.go:32] "PullImage from image service failed" err="rpc error: code = Unknown desc = unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/c2/c2e50d184eed6dc4c810c7883bd3854d3a15078b82f830637490b9bd1fd9ef20?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260712%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260712T160001Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=71b9e1d137353f0b58c92cd1e904aba8dee9abb62d67c7250f89035793884fea®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=ubi9----ubi-minimal&akamai_signature=exp=1783872901~hmac=ba289ef9c5cfbfd71dadc77fbd0f9b35e1c9cdfb8146601a0468c2d01f6dcfd8\": EOF; artifact err: provided artifact is a container image" image="registry.redhat.io/ubi9/ubi-minimal:9.7" Jul 12 16:00:01.554383 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:00:01.554005 2567 kuberuntime_manager.go:1459] "Unhandled Error" err="container cleanup start failed in pod maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp_opendatahub(27d95005-ac01-42e6-a885-0861221d23cc): ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: Get \"https://cdn01.quay.io/quayio-production-s3/sha256/c2/c2e50d184eed6dc4c810c7883bd3854d3a15078b82f830637490b9bd1fd9ef20?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260712%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260712T160001Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=71b9e1d137353f0b58c92cd1e904aba8dee9abb62d67c7250f89035793884fea®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=ubi9----ubi-minimal&akamai_signature=exp=1783872901~hmac=ba289ef9c5cfbfd71dadc77fbd0f9b35e1c9cdfb8146601a0468c2d01f6dcfd8\": EOF; artifact err: provided artifact is a container image" logger="UnhandledError" Jul 12 16:00:01.554383 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:00:01.554078 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with ErrImagePull: \"unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/c2/c2e50d184eed6dc4c810c7883bd3854d3a15078b82f830637490b9bd1fd9ef20?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260712%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260712T160001Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=71b9e1d137353f0b58c92cd1e904aba8dee9abb62d67c7250f89035793884fea®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=ubi9----ubi-minimal&akamai_signature=exp=1783872901~hmac=ba289ef9c5cfbfd71dadc77fbd0f9b35e1c9cdfb8146601a0468c2d01f6dcfd8\\\": EOF; artifact err: provided artifact is a container image\"" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" podUID="27d95005-ac01-42e6-a885-0861221d23cc" Jul 12 16:00:01.611738 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:00:01.611704 2567 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with ImagePullBackOff: \"Back-off pulling image \\\"registry.redhat.io/ubi9/ubi-minimal:9.7\\\": ErrImagePull: unable to pull image or OCI artifact: pull image err: copying system image from manifest list: parsing image configuration: Get \\\"https://cdn01.quay.io/quayio-production-s3/sha256/c2/c2e50d184eed6dc4c810c7883bd3854d3a15078b82f830637490b9bd1fd9ef20?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIATAAF2YHTGR23ZTE6%2F20260712%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20260712T160001Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=71b9e1d137353f0b58c92cd1e904aba8dee9abb62d67c7250f89035793884fea®ion=us-east-1&namespace=redhat-prod&username=redhat-prod+registry_proxy&repo_name=ubi9----ubi-minimal&akamai_signature=exp=1783872901~hmac=ba289ef9c5cfbfd71dadc77fbd0f9b35e1c9cdfb8146601a0468c2d01f6dcfd8\\\": EOF; artifact err: provided artifact is a container image\"" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" podUID="27d95005-ac01-42e6-a885-0861221d23cc" Jul 12 16:00:16.591214 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:16.591164 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 16:00:16.680695 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:16.680663 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" event={"ID":"27d95005-ac01-42e6-a885-0861221d23cc","Type":"ContainerStarted","Data":"cc5c0156cbd88f529cd4276ef97f36cf1a3729deba7a166f34cf89e3b6102511"} Jul 12 16:00:16.697254 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:16.697203 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" podStartSLOduration=0.991686978 podStartE2EDuration="16.697187088s" totalImagesPullingTime="15.70550011s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 16:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=1 observedRunningTime="2026-07-12 16:00:16.695847244 +0000 UTC m=+2394.404654864" watchObservedRunningTime="2026-07-12 16:00:16.697187088 +0000 UTC m=+2394.405994681" Jul 12 16:00:17.686949 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:17.686915 2567 generic.go:356] "Generic (PLEG): container finished" podID="27d95005-ac01-42e6-a885-0861221d23cc" containerID="cc5c0156cbd88f529cd4276ef97f36cf1a3729deba7a166f34cf89e3b6102511" exitCode=0 Jul 12 16:00:17.687351 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:17.686957 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" event={"ID":"27d95005-ac01-42e6-a885-0861221d23cc","Type":"ContainerDied","Data":"cc5c0156cbd88f529cd4276ef97f36cf1a3729deba7a166f34cf89e3b6102511"} Jul 12 16:00:18.827204 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:18.827182 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:18.835153 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:18.835131 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wphv9\" (UniqueName: \"kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9\") pod \"27d95005-ac01-42e6-a885-0861221d23cc\" (UID: \"27d95005-ac01-42e6-a885-0861221d23cc\") " Jul 12 16:00:18.837331 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:18.837308 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9" (OuterVolumeSpecName: "kube-api-access-wphv9") pod "27d95005-ac01-42e6-a885-0861221d23cc" (UID: "27d95005-ac01-42e6-a885-0861221d23cc"). InnerVolumeSpecName "kube-api-access-wphv9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:00:18.936482 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:18.936451 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wphv9\" (UniqueName: \"kubernetes.io/projected/27d95005-ac01-42e6-a885-0861221d23cc-kube-api-access-wphv9\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:19.704321 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:19.704292 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" Jul 12 16:00:19.704321 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:19.704307 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp" event={"ID":"27d95005-ac01-42e6-a885-0861221d23cc","Type":"ContainerDied","Data":"a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8"} Jul 12 16:00:19.704542 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:19.704374 2567 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a2965abb96c2e1d2a374057bf97c9dd3987ff7012e4b655ece15ef7b01e364f8" Jul 12 16:00:22.914729 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:22.914702 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 16:00:22.921707 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:22.921687 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 16:00:22.924024 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:22.924007 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 16:00:22.930471 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:22.930454 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 16:00:29.596142 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.596110 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:00:29.596515 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.596501 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27d95005-ac01-42e6-a885-0861221d23cc" containerName="cleanup" Jul 12 16:00:29.596571 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.596517 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="27d95005-ac01-42e6-a885-0861221d23cc" containerName="cleanup" Jul 12 16:00:29.596606 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.596596 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="27d95005-ac01-42e6-a885-0861221d23cc" containerName="cleanup" Jul 12 16:00:29.599550 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.599530 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.602189 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.602171 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a99179\"/\"sub-test-model-a99179-kserve-self-signed-certs\"" Jul 12 16:00:29.609922 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.609899 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:00:29.631316 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631293 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.631434 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631325 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtdcv\" (UniqueName: \"kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.631434 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631355 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.631515 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631453 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.631515 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631477 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.631515 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.631493 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.731889 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.731849 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.731889 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.731890 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732174 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.731948 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732174 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.731971 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtdcv\" (UniqueName: \"kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732174 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.732013 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732174 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.732107 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732373 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.732324 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732373 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.732340 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.732482 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.732419 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.734316 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.734294 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.734601 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.734583 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.740114 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.740093 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtdcv\" (UniqueName: \"kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv\") pod \"sub-test-model-a99179-kserve-579ff4456c-fr2br\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:29.910620 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:29.910520 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:30.048773 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:30.048741 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:00:30.051668 ip-10-0-136-30 kubenswrapper[2567]: W0712 16:00:30.051635 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9f69294d_3602_4fa4_aa5b_4ebe0b476406.slice/crio-87d48ad841aba7f3aad45018502034bc9668e471a5c7fac6cfb659b821458f28 WatchSource:0}: Error finding container 87d48ad841aba7f3aad45018502034bc9668e471a5c7fac6cfb659b821458f28: Status 404 returned error can't find the container with id 87d48ad841aba7f3aad45018502034bc9668e471a5c7fac6cfb659b821458f28 Jul 12 16:00:30.758408 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:30.758375 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" event={"ID":"9f69294d-3602-4fa4-aa5b-4ebe0b476406","Type":"ContainerStarted","Data":"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f"} Jul 12 16:00:30.758408 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:30.758419 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" event={"ID":"9f69294d-3602-4fa4-aa5b-4ebe0b476406","Type":"ContainerStarted","Data":"87d48ad841aba7f3aad45018502034bc9668e471a5c7fac6cfb659b821458f28"} Jul 12 16:00:30.776151 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:30.776104 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" podStartSLOduration=1.7760899129999999 podStartE2EDuration="1.776089913s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 16:00:29 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 16:00:30.774944665 +0000 UTC m=+2408.483752318" watchObservedRunningTime="2026-07-12 16:00:30.776089913 +0000 UTC m=+2408.484897506" Jul 12 16:00:31.614833 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.614794 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" podUID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" containerName="main" containerID="cri-o://96d8f2fc60ce791f5ec1d977841e9ec12fd2af724beb166c028cd03d804fc10d" gracePeriod=45 Jul 12 16:00:31.765142 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.765111 2567 generic.go:356] "Generic (PLEG): container finished" podID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" containerID="96d8f2fc60ce791f5ec1d977841e9ec12fd2af724beb166c028cd03d804fc10d" exitCode=0 Jul 12 16:00:31.765511 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.765190 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" event={"ID":"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2","Type":"ContainerDied","Data":"96d8f2fc60ce791f5ec1d977841e9ec12fd2af724beb166c028cd03d804fc10d"} Jul 12 16:00:31.873126 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.873068 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 16:00:31.952761 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952728 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.952938 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952824 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.952938 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952854 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.952938 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952888 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rwbpm\" (UniqueName: \"kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.952938 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952923 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.953199 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.952955 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir\") pod \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\" (UID: \"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2\") " Jul 12 16:00:31.953199 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.953017 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache" (OuterVolumeSpecName: "model-cache") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:00:31.953320 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.953252 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home" (OuterVolumeSpecName: "home") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:00:31.953320 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.953263 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:00:31.953426 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.953344 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-model-cache\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:31.955244 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.955207 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm" (OuterVolumeSpecName: "dshm") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:00:31.955244 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.955227 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 16:00:31.955244 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:31.955237 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm" (OuterVolumeSpecName: "kube-api-access-rwbpm") pod "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" (UID: "e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2"). InnerVolumeSpecName "kube-api-access-rwbpm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:00:32.054156 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.054121 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rwbpm\" (UniqueName: \"kubernetes.io/projected/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-kube-api-access-rwbpm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:32.054156 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.054151 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-home\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:32.054156 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.054161 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tmp-dir\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:32.054376 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.054169 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-dshm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:32.054376 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.054177 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2-tls-certs\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:00:32.771543 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.771507 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" event={"ID":"e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2","Type":"ContainerDied","Data":"8c0bac32f6be62a2873ba33781c7457219135b0f314fcd61bf6bb5b5fc9e2e4e"} Jul 12 16:00:32.771543 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.771535 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg" Jul 12 16:00:32.772039 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.771563 2567 scope.go:117] "RemoveContainer" containerID="96d8f2fc60ce791f5ec1d977841e9ec12fd2af724beb166c028cd03d804fc10d" Jul 12 16:00:32.793182 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.793155 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 16:00:32.794841 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.794816 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/auth-test-model-a99179-kserve-74c9c54775-7mplg"] Jul 12 16:00:32.847313 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:32.847277 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" path="/var/lib/kubelet/pods/e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2/volumes" Jul 12 16:00:39.911159 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:39.911118 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:39.911578 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:39.911383 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:39.923810 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:39.923786 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:00:40.823185 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:00:40.823158 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:01:39.818652 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:39.818614 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:01:54.840038 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:54.839973 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" podUID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" containerName="main" containerID="cri-o://84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f" gracePeriod=45 Jul 12 16:01:55.087549 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.087524 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:01:55.120736 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120659 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.120736 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120708 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.120945 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120753 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.120945 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120780 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.120945 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120794 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.120945 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120820 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dtdcv\" (UniqueName: \"kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv\") pod \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\" (UID: \"9f69294d-3602-4fa4-aa5b-4ebe0b476406\") " Jul 12 16:01:55.121188 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.120938 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:01:55.121188 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.121073 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tmp-dir\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:55.121188 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.121082 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache" (OuterVolumeSpecName: "model-cache") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:01:55.121188 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.121122 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home" (OuterVolumeSpecName: "home") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:01:55.123069 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.123015 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm" (OuterVolumeSpecName: "dshm") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:01:55.123168 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.123093 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 16:01:55.123168 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.123132 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv" (OuterVolumeSpecName: "kube-api-access-dtdcv") pod "9f69294d-3602-4fa4-aa5b-4ebe0b476406" (UID: "9f69294d-3602-4fa4-aa5b-4ebe0b476406"). InnerVolumeSpecName "kube-api-access-dtdcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:01:55.157678 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.157648 2567 generic.go:356] "Generic (PLEG): container finished" podID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" containerID="84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f" exitCode=0 Jul 12 16:01:55.157806 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.157693 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" Jul 12 16:01:55.157806 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.157732 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" event={"ID":"9f69294d-3602-4fa4-aa5b-4ebe0b476406","Type":"ContainerDied","Data":"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f"} Jul 12 16:01:55.157806 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.157776 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br" event={"ID":"9f69294d-3602-4fa4-aa5b-4ebe0b476406","Type":"ContainerDied","Data":"87d48ad841aba7f3aad45018502034bc9668e471a5c7fac6cfb659b821458f28"} Jul 12 16:01:55.157806 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.157800 2567 scope.go:117] "RemoveContainer" containerID="84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f" Jul 12 16:01:55.166497 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.166478 2567 scope.go:117] "RemoveContainer" containerID="84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f" Jul 12 16:01:55.166748 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:01:55.166731 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f\": container with ID starting with 84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f not found: ID does not exist" containerID="84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f" Jul 12 16:01:55.166815 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.166758 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f"} err="failed to get container status \"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f\": rpc error: code = NotFound desc = could not find container \"84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f\": container with ID starting with 84042efb24b75385cd477e2b836c0fbbd818b5ebf1f323d3187cf4bf2141b60f not found: ID does not exist" Jul 12 16:01:55.179691 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.179648 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:01:55.181468 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.181445 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a99179/sub-test-model-a99179-kserve-579ff4456c-fr2br"] Jul 12 16:01:55.221664 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.221638 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-dshm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:55.221664 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.221666 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-model-cache\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:55.221796 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.221677 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9f69294d-3602-4fa4-aa5b-4ebe0b476406-home\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:55.221796 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.221686 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dtdcv\" (UniqueName: \"kubernetes.io/projected/9f69294d-3602-4fa4-aa5b-4ebe0b476406-kube-api-access-dtdcv\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:55.221796 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:55.221694 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9f69294d-3602-4fa4-aa5b-4ebe0b476406-tls-certs\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:01:56.846734 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:01:56.846691 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" path="/var/lib/kubelet/pods/9f69294d-3602-4fa4-aa5b-4ebe0b476406/volumes" Jul 12 16:02:13.492285 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492198 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:02:13.492781 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492760 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" containerName="main" Jul 12 16:02:13.492862 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492785 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" containerName="main" Jul 12 16:02:13.492862 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492804 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" containerName="main" Jul 12 16:02:13.492862 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492814 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" containerName="main" Jul 12 16:02:13.493028 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492935 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="9f69294d-3602-4fa4-aa5b-4ebe0b476406" containerName="main" Jul 12 16:02:13.493028 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.492956 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="e03ae7e2-8c96-4c11-9be5-ad3c99b78aa2" containerName="main" Jul 12 16:02:13.496257 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.496239 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.498790 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.498767 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3da434\"/\"kube-root-ca.crt\"" Jul 12 16:02:13.500205 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.500183 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3da434\"/\"default-dockercfg-njf7m\"" Jul 12 16:02:13.500205 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.500202 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-3da434\"/\"openshift-service-ca.crt\"" Jul 12 16:02:13.500435 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.500238 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-3da434\"/\"rate-test-model-3da434-kserve-self-signed-certs\"" Jul 12 16:02:13.502326 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.502301 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:02:13.587592 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587560 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.587769 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587623 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.587769 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587649 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.587769 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587669 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tmqw\" (UniqueName: \"kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.587769 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587704 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.587769 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.587742 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689138 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689100 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689328 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689163 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689328 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689206 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689328 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689263 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689328 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689290 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689328 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689318 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tmqw\" (UniqueName: \"kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689672 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689630 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689795 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689671 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.689795 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.689708 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.691486 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.691464 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.692073 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.692034 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.697257 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.697228 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tmqw\" (UniqueName: \"kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw\") pod \"rate-test-model-3da434-kserve-57678c9d95-7h9w4\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.806811 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.806782 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:13.943135 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:13.943109 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:02:13.944907 ip-10-0-136-30 kubenswrapper[2567]: W0712 16:02:13.944880 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2399c6a_b32e_4610_8a53_5f2298a815af.slice/crio-c8c65532b2540bd55f2f9d2f02980d81e2db1b0e32d2c010f6eade3805cb5a44 WatchSource:0}: Error finding container c8c65532b2540bd55f2f9d2f02980d81e2db1b0e32d2c010f6eade3805cb5a44: Status 404 returned error can't find the container with id c8c65532b2540bd55f2f9d2f02980d81e2db1b0e32d2c010f6eade3805cb5a44 Jul 12 16:02:14.255024 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:14.254934 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" event={"ID":"d2399c6a-b32e-4610-8a53-5f2298a815af","Type":"ContainerStarted","Data":"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3"} Jul 12 16:02:14.255024 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:14.254983 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" event={"ID":"d2399c6a-b32e-4610-8a53-5f2298a815af","Type":"ContainerStarted","Data":"c8c65532b2540bd55f2f9d2f02980d81e2db1b0e32d2c010f6eade3805cb5a44"} Jul 12 16:02:14.272176 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:14.272132 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" podStartSLOduration=1.272116824 podStartE2EDuration="1.272116824s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 16:02:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 16:02:14.270408158 +0000 UTC m=+2511.979215777" watchObservedRunningTime="2026-07-12 16:02:14.272116824 +0000 UTC m=+2511.980924421" Jul 12 16:02:23.807788 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:23.807750 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:23.808291 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:23.808019 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:23.820368 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:23.820341 2567 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:24.317034 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:24.317006 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:02:55.179957 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:02:55.179924 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:03:10.201216 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.201128 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" podUID="d2399c6a-b32e-4610-8a53-5f2298a815af" containerName="main" containerID="cri-o://23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3" gracePeriod=45 Jul 12 16:03:10.451757 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.451704 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:03:10.530312 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.530276 2567 generic.go:356] "Generic (PLEG): container finished" podID="d2399c6a-b32e-4610-8a53-5f2298a815af" containerID="23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3" exitCode=0 Jul 12 16:03:10.530469 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.530332 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" Jul 12 16:03:10.530469 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.530356 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" event={"ID":"d2399c6a-b32e-4610-8a53-5f2298a815af","Type":"ContainerDied","Data":"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3"} Jul 12 16:03:10.530469 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.530396 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4" event={"ID":"d2399c6a-b32e-4610-8a53-5f2298a815af","Type":"ContainerDied","Data":"c8c65532b2540bd55f2f9d2f02980d81e2db1b0e32d2c010f6eade3805cb5a44"} Jul 12 16:03:10.530469 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.530413 2567 scope.go:117] "RemoveContainer" containerID="23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3" Jul 12 16:03:10.538942 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.538926 2567 scope.go:117] "RemoveContainer" containerID="23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3" Jul 12 16:03:10.539221 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:03:10.539197 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3\": container with ID starting with 23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3 not found: ID does not exist" containerID="23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3" Jul 12 16:03:10.539274 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.539224 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3"} err="failed to get container status \"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3\": rpc error: code = NotFound desc = could not find container \"23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3\": container with ID starting with 23a8efcb6de680b8033dff00aed6995dc63c898ffe690d7030ecf5cb227289b3 not found: ID does not exist" Jul 12 16:03:10.604108 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604086 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604231 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604120 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604231 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604166 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604231 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604186 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604373 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604246 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4tmqw\" (UniqueName: \"kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604373 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604317 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm\") pod \"d2399c6a-b32e-4610-8a53-5f2298a815af\" (UID: \"d2399c6a-b32e-4610-8a53-5f2298a815af\") " Jul 12 16:03:10.604373 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604355 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache" (OuterVolumeSpecName: "model-cache") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:10.604525 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604441 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home" (OuterVolumeSpecName: "home") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:10.604588 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604521 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:10.604649 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604635 2567 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-model-cache\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.604649 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604646 2567 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-home\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.604728 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.604670 2567 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-tmp-dir\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.606575 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.606552 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm" (OuterVolumeSpecName: "dshm") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 16:03:10.606663 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.606574 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 16:03:10.606663 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.606598 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw" (OuterVolumeSpecName: "kube-api-access-4tmqw") pod "d2399c6a-b32e-4610-8a53-5f2298a815af" (UID: "d2399c6a-b32e-4610-8a53-5f2298a815af"). InnerVolumeSpecName "kube-api-access-4tmqw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:10.705837 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.705771 2567 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d2399c6a-b32e-4610-8a53-5f2298a815af-tls-certs\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.705837 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.705799 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4tmqw\" (UniqueName: \"kubernetes.io/projected/d2399c6a-b32e-4610-8a53-5f2298a815af-kube-api-access-4tmqw\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.705837 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.705809 2567 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d2399c6a-b32e-4610-8a53-5f2298a815af-dshm\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:10.851869 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.851839 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:03:10.855188 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:10.855167 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-3da434/rate-test-model-3da434-kserve-57678c9d95-7h9w4"] Jul 12 16:03:12.845927 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:12.845893 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d2399c6a-b32e-4610-8a53-5f2298a815af" path="/var/lib/kubelet/pods/d2399c6a-b32e-4610-8a53-5f2298a815af/volumes" Jul 12 16:03:39.617379 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.617340 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 16:03:39.617842 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.617603 2567 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" podUID="b4c80bc6-d145-47bf-86e9-468491e9d872" containerName="maas-api" containerID="cri-o://cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba" gracePeriod=30 Jul 12 16:03:39.639977 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.639933 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp"] Jul 12 16:03:39.644744 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.644722 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-shared-b-3da434-29731200-swqjp"] Jul 12 16:03:39.867106 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.867080 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 16:03:39.948680 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.948589 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wk9f8\" (UniqueName: \"kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8\") pod \"b4c80bc6-d145-47bf-86e9-468491e9d872\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " Jul 12 16:03:39.948680 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.948671 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca\") pod \"b4c80bc6-d145-47bf-86e9-468491e9d872\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " Jul 12 16:03:39.948871 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.948840 2567 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls\") pod \"b4c80bc6-d145-47bf-86e9-468491e9d872\" (UID: \"b4c80bc6-d145-47bf-86e9-468491e9d872\") " Jul 12 16:03:39.949033 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.949010 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b4c80bc6-d145-47bf-86e9-468491e9d872" (UID: "b4c80bc6-d145-47bf-86e9-468491e9d872"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 16:03:39.949255 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.949231 2567 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4c80bc6-d145-47bf-86e9-468491e9d872-openshift-service-ca\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:39.950940 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.950916 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8" (OuterVolumeSpecName: "kube-api-access-wk9f8") pod "b4c80bc6-d145-47bf-86e9-468491e9d872" (UID: "b4c80bc6-d145-47bf-86e9-468491e9d872"). InnerVolumeSpecName "kube-api-access-wk9f8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 16:03:39.951067 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:39.950992 2567 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b4c80bc6-d145-47bf-86e9-468491e9d872" (UID: "b4c80bc6-d145-47bf-86e9-468491e9d872"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 16:03:40.050271 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.050232 2567 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wk9f8\" (UniqueName: \"kubernetes.io/projected/b4c80bc6-d145-47bf-86e9-468491e9d872-kube-api-access-wk9f8\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:40.050271 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.050265 2567 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b4c80bc6-d145-47bf-86e9-468491e9d872-maas-api-tls\") on node \"ip-10-0-136-30.ec2.internal\" DevicePath \"\"" Jul 12 16:03:40.675873 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.675837 2567 generic.go:356] "Generic (PLEG): container finished" podID="b4c80bc6-d145-47bf-86e9-468491e9d872" containerID="cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba" exitCode=0 Jul 12 16:03:40.676323 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.675895 2567 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" Jul 12 16:03:40.676323 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.675912 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" event={"ID":"b4c80bc6-d145-47bf-86e9-468491e9d872","Type":"ContainerDied","Data":"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba"} Jul 12 16:03:40.676323 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.675988 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx" event={"ID":"b4c80bc6-d145-47bf-86e9-468491e9d872","Type":"ContainerDied","Data":"97a36494fe5fe23072b96236943119b089e4cba19d74f19fb6e3873c054e6d2d"} Jul 12 16:03:40.676323 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.676011 2567 scope.go:117] "RemoveContainer" containerID="cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba" Jul 12 16:03:40.685549 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.685529 2567 scope.go:117] "RemoveContainer" containerID="cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba" Jul 12 16:03:40.685859 ip-10-0-136-30 kubenswrapper[2567]: E0712 16:03:40.685839 2567 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba\": container with ID starting with cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba not found: ID does not exist" containerID="cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba" Jul 12 16:03:40.685924 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.685869 2567 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba"} err="failed to get container status \"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba\": rpc error: code = NotFound desc = could not find container \"cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba\": container with ID starting with cd3aa11c2fb9a637a3a90e72883a6fe16c789a002175cf0479eca279e8a9d7ba not found: ID does not exist" Jul 12 16:03:40.697536 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.697509 2567 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 16:03:40.699226 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.699199 2567 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-3da434-7b8df9b5f8-npcwx"] Jul 12 16:03:40.846010 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.845973 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27d95005-ac01-42e6-a885-0861221d23cc" path="/var/lib/kubelet/pods/27d95005-ac01-42e6-a885-0861221d23cc/volumes" Jul 12 16:03:40.846384 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:40.846370 2567 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4c80bc6-d145-47bf-86e9-468491e9d872" path="/var/lib/kubelet/pods/b4c80bc6-d145-47bf-86e9-468491e9d872/volumes" Jul 12 16:03:46.156587 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:46.156497 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-df97db75-wqb4m_c99e4b48-c433-4f70-b2e3-df9ddcd1f859/authorino/0.log" Jul 12 16:03:51.477455 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:51.477430 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-992w4_ee1fa10f-0d01-48c3-ab56-790d581beb5e/copy-manifests/0.log" Jul 12 16:03:51.483726 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:51.483703 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-992w4_ee1fa10f-0d01-48c3-ab56-790d581beb5e/manager/0.log" Jul 12 16:03:51.822728 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:51.822696 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qbdk6_4039edfc-5466-49ed-807c-90f8215510d0/postgres/0.log" Jul 12 16:03:53.079095 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:53.079022 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-df97db75-wqb4m_c99e4b48-c433-4f70-b2e3-df9ddcd1f859/authorino/0.log" Jul 12 16:03:53.633789 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:53.633757 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l8l2b_88aa582d-fda9-4fc1-877f-0ad81b1fdef0/manager/0.log" Jul 12 16:03:54.519515 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:54.519479 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fb64c5677-89cjz_96b55013-d5e3-4886-b981-c8fc69eab194/kube-auth-proxy/0.log" Jul 12 16:03:55.401974 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:55.401937 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-8bjcr_839c3860-eb8b-426a-b6ee-18f25b3e9e88/main/0.log" Jul 12 16:03:55.733395 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:55.733319 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-v24cw_1725fb5c-fddc-4f0e-9729-c741adf1c535/main/0.log" Jul 12 16:03:59.896715 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.896686 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nth9q/must-gather-2882m"] Jul 12 16:03:59.897096 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897021 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d2399c6a-b32e-4610-8a53-5f2298a815af" containerName="main" Jul 12 16:03:59.897096 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897031 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="d2399c6a-b32e-4610-8a53-5f2298a815af" containerName="main" Jul 12 16:03:59.897096 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897087 2567 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4c80bc6-d145-47bf-86e9-468491e9d872" containerName="maas-api" Jul 12 16:03:59.897096 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897094 2567 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4c80bc6-d145-47bf-86e9-468491e9d872" containerName="maas-api" Jul 12 16:03:59.897235 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897150 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="d2399c6a-b32e-4610-8a53-5f2298a815af" containerName="main" Jul 12 16:03:59.897235 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.897160 2567 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4c80bc6-d145-47bf-86e9-468491e9d872" containerName="maas-api" Jul 12 16:03:59.903975 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.902323 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:03:59.906426 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.906399 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nth9q\"/\"openshift-service-ca.crt\"" Jul 12 16:03:59.906575 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.906401 2567 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-nth9q\"/\"kube-root-ca.crt\"" Jul 12 16:03:59.906575 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.906401 2567 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-nth9q\"/\"default-dockercfg-p7gsx\"" Jul 12 16:03:59.913690 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:03:59.913666 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/must-gather-2882m"] Jul 12 16:04:00.037451 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.037399 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7lfb\" (UniqueName: \"kubernetes.io/projected/0d5c5969-5aed-4e1e-8565-603628196416-kube-api-access-t7lfb\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.037451 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.037461 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d5c5969-5aed-4e1e-8565-603628196416-must-gather-output\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.138243 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.138201 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7lfb\" (UniqueName: \"kubernetes.io/projected/0d5c5969-5aed-4e1e-8565-603628196416-kube-api-access-t7lfb\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.138243 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.138250 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d5c5969-5aed-4e1e-8565-603628196416-must-gather-output\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.138536 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.138520 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/0d5c5969-5aed-4e1e-8565-603628196416-must-gather-output\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.148890 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.148822 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7lfb\" (UniqueName: \"kubernetes.io/projected/0d5c5969-5aed-4e1e-8565-603628196416-kube-api-access-t7lfb\") pod \"must-gather-2882m\" (UID: \"0d5c5969-5aed-4e1e-8565-603628196416\") " pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.213705 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.213665 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/must-gather-2882m" Jul 12 16:04:00.354502 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.354472 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/must-gather-2882m"] Jul 12 16:04:00.356592 ip-10-0-136-30 kubenswrapper[2567]: W0712 16:04:00.356560 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0d5c5969_5aed_4e1e_8565_603628196416.slice/crio-b570886622d17b923b4f6e2801a08c0e9b67c56f741565fdc847fb4bd030472f WatchSource:0}: Error finding container b570886622d17b923b4f6e2801a08c0e9b67c56f741565fdc847fb4bd030472f: Status 404 returned error can't find the container with id b570886622d17b923b4f6e2801a08c0e9b67c56f741565fdc847fb4bd030472f Jul 12 16:04:00.772073 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:00.772007 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/must-gather-2882m" event={"ID":"0d5c5969-5aed-4e1e-8565-603628196416","Type":"ContainerStarted","Data":"b570886622d17b923b4f6e2801a08c0e9b67c56f741565fdc847fb4bd030472f"} Jul 12 16:04:01.780150 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:01.780040 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/must-gather-2882m" event={"ID":"0d5c5969-5aed-4e1e-8565-603628196416","Type":"ContainerStarted","Data":"c16c19c2c116bc15935b4c436db8b2da9058809755b7927d1d2f7757456b8f30"} Jul 12 16:04:01.780150 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:01.780123 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/must-gather-2882m" event={"ID":"0d5c5969-5aed-4e1e-8565-603628196416","Type":"ContainerStarted","Data":"3ccaa99fd9d412dba0f14087d0d3507bb38588a8d14a4ac6c34eed36a14d62e8"} Jul 12 16:04:01.797138 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:01.797073 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nth9q/must-gather-2882m" podStartSLOduration=1.894588766 podStartE2EDuration="2.797032443s" totalImagesPullingTime="902.443677ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 16:03:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 16:04:01.793953285 +0000 UTC m=+2619.502760903" watchObservedRunningTime="2026-07-12 16:04:01.797032443 +0000 UTC m=+2619.505840038" Jul 12 16:04:02.759745 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:02.759704 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-bb2p6_4676e89d-0cdb-4fb5-9871-6daebefa4b95/global-pull-secret-syncer/0.log" Jul 12 16:04:02.938586 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:02.938558 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tnshv_29a191cb-53c1-4bd2-accd-e848a25fa46e/konnectivity-agent/0.log" Jul 12 16:04:03.010909 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:03.010831 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-136-30.ec2.internal_5c6eb2977c26623da22c14690f0e26ab/haproxy/0.log" Jul 12 16:04:08.438577 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:08.437931 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-df97db75-wqb4m_c99e4b48-c433-4f70-b2e3-df9ddcd1f859/authorino/0.log" Jul 12 16:04:08.788970 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:08.788887 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-l8l2b_88aa582d-fda9-4fc1-877f-0ad81b1fdef0/manager/0.log" Jul 12 16:04:10.754532 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.754417 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-jdjc7_44f318e0-6b39-4c0d-b348-afe3f1e3c05b/cluster-monitoring-operator/0.log" Jul 12 16:04:10.784028 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.783992 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ff8fp_49605801-07e1-4436-bd28-fa5d67213494/kube-state-metrics/0.log" Jul 12 16:04:10.806089 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.806035 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ff8fp_49605801-07e1-4436-bd28-fa5d67213494/kube-rbac-proxy-main/0.log" Jul 12 16:04:10.838944 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.838738 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-ff8fp_49605801-07e1-4436-bd28-fa5d67213494/kube-rbac-proxy-self/0.log" Jul 12 16:04:10.871072 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.868303 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5ccc6b58cf-zjcw5_2bcb06f4-dc4c-423e-80f7-9cdef0899fd4/metrics-server/0.log" Jul 12 16:04:10.999908 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:10.999874 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j5xx_56a2ec36-946b-4d54-96d9-f88cd45d1d93/node-exporter/0.log" Jul 12 16:04:11.018637 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.018538 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j5xx_56a2ec36-946b-4d54-96d9-f88cd45d1d93/kube-rbac-proxy/0.log" Jul 12 16:04:11.038915 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.038881 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-7j5xx_56a2ec36-946b-4d54-96d9-f88cd45d1d93/init-textfile/0.log" Jul 12 16:04:11.158151 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.158106 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-cglmk_5d14f178-0fd1-4143-a142-68b649c2b61b/kube-rbac-proxy-main/0.log" Jul 12 16:04:11.182744 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.182710 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-cglmk_5d14f178-0fd1-4143-a142-68b649c2b61b/kube-rbac-proxy-self/0.log" Jul 12 16:04:11.206895 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.206863 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-cglmk_5d14f178-0fd1-4143-a142-68b649c2b61b/openshift-state-metrics/0.log" Jul 12 16:04:11.246857 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.246826 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/prometheus/0.log" Jul 12 16:04:11.267417 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.267390 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/config-reloader/0.log" Jul 12 16:04:11.287957 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.287876 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/thanos-sidecar/0.log" Jul 12 16:04:11.313574 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.313546 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/kube-rbac-proxy-web/0.log" Jul 12 16:04:11.374952 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.374853 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/kube-rbac-proxy/0.log" Jul 12 16:04:11.437452 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.437416 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/kube-rbac-proxy-thanos/0.log" Jul 12 16:04:11.461385 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.461360 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_5739e66e-158e-4554-9220-31b04d16f960/init-config-reloader/0.log" Jul 12 16:04:11.539126 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:11.538423 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-7rd6r_d78631af-aba3-42c3-aa6c-0c3721d050e8/prometheus-operator-admission-webhook/0.log" Jul 12 16:04:13.515947 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.515896 2567 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr"] Jul 12 16:04:13.522282 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.522243 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.528524 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.528488 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr"] Jul 12 16:04:13.583931 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.583888 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-lib-modules\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.584233 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.584209 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-podres\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.584351 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.584336 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-sys\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.584485 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.584471 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vbbv\" (UniqueName: \"kubernetes.io/projected/7278d762-2af8-4a1b-aead-e98e6fc4a043-kube-api-access-5vbbv\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.584578 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.584568 2567 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-proc\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.615584 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.615548 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/1.log" Jul 12 16:04:13.622173 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.622138 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-lk5sx_88c42517-0b00-4aae-8dde-6805b3dcb6b2/console-operator/2.log" Jul 12 16:04:13.686011 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.685978 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-podres\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686219 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686185 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-podres\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686296 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686217 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-sys\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686296 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686287 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vbbv\" (UniqueName: \"kubernetes.io/projected/7278d762-2af8-4a1b-aead-e98e6fc4a043-kube-api-access-5vbbv\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686401 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686296 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-sys\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686401 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686323 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-proc\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686505 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686435 2567 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-lib-modules\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686555 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686494 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-proc\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.686640 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.686601 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7278d762-2af8-4a1b-aead-e98e6fc4a043-lib-modules\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.694894 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.694865 2567 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vbbv\" (UniqueName: \"kubernetes.io/projected/7278d762-2af8-4a1b-aead-e98e6fc4a043-kube-api-access-5vbbv\") pod \"perf-node-gather-daemonset-p9gxr\" (UID: \"7278d762-2af8-4a1b-aead-e98e6fc4a043\") " pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:13.837850 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:13.837805 2567 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:14.029000 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.028905 2567 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr"] Jul 12 16:04:14.031666 ip-10-0-136-30 kubenswrapper[2567]: W0712 16:04:14.031636 2567 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod7278d762_2af8_4a1b_aead_e98e6fc4a043.slice/crio-0bc31a2a84aa20db11702fd864814750540fb01cc55da9063a356f8b1037037a WatchSource:0}: Error finding container 0bc31a2a84aa20db11702fd864814750540fb01cc55da9063a356f8b1037037a: Status 404 returned error can't find the container with id 0bc31a2a84aa20db11702fd864814750540fb01cc55da9063a356f8b1037037a Jul 12 16:04:14.178093 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.178060 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-fmjc5_314047f5-49d6-4509-ab39-b49f0e4e6ca2/download-server/0.log" Jul 12 16:04:14.716817 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.716787 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-c88vb_65c4e4d8-19d5-497c-96de-abfb7986b36a/volume-data-source-validator/0.log" Jul 12 16:04:14.873470 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.873431 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" event={"ID":"7278d762-2af8-4a1b-aead-e98e6fc4a043","Type":"ContainerStarted","Data":"1bd6e084c72a6194d0f728c6258fc7a648754f1a8a356bed2c7d2996d4c24ac8"} Jul 12 16:04:14.873684 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.873492 2567 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" event={"ID":"7278d762-2af8-4a1b-aead-e98e6fc4a043","Type":"ContainerStarted","Data":"0bc31a2a84aa20db11702fd864814750540fb01cc55da9063a356f8b1037037a"} Jul 12 16:04:14.873684 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.873550 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:14.889824 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:14.889774 2567 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" podStartSLOduration=1.8897582609999999 podStartE2EDuration="1.889758261s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 16:04:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 16:04:14.88815873 +0000 UTC m=+2632.596966326" watchObservedRunningTime="2026-07-12 16:04:14.889758261 +0000 UTC m=+2632.598565855" Jul 12 16:04:15.684610 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:15.684583 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5xzvm_3a635a9e-3dbc-40c4-b4ca-7240719f4e5c/dns/0.log" Jul 12 16:04:15.706778 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:15.706745 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5xzvm_3a635a9e-3dbc-40c4-b4ca-7240719f4e5c/kube-rbac-proxy/0.log" Jul 12 16:04:15.814308 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:15.814279 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5xx64_65cba0d8-d7a9-4d40-893b-b082b5f72ae4/dns-node-resolver/0.log" Jul 12 16:04:16.474560 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:16.474531 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7g55n_ec638ef4-7065-4b9a-9ac7-102a3ad10865/node-ca/0.log" Jul 12 16:04:17.639690 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:17.639659 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fb64c5677-89cjz_96b55013-d5e3-4886-b981-c8fc69eab194/kube-auth-proxy/0.log" Jul 12 16:04:18.458825 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:18.458791 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-g5ds8_1d63f4ab-9cfd-46d1-9dc5-0da56873da41/serve-healthcheck-canary/0.log" Jul 12 16:04:19.070424 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:19.070391 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2tkh_406610d9-4c13-4368-9059-776813671305/kube-rbac-proxy/0.log" Jul 12 16:04:19.089579 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:19.089554 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2tkh_406610d9-4c13-4368-9059-776813671305/exporter/0.log" Jul 12 16:04:19.110339 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:19.110310 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-m2tkh_406610d9-4c13-4368-9059-776813671305/extractor/0.log" Jul 12 16:04:20.892245 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:20.892215 2567 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-nth9q/perf-node-gather-daemonset-p9gxr" Jul 12 16:04:21.577416 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:21.577362 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-992w4_ee1fa10f-0d01-48c3-ab56-790d581beb5e/manager/0.log" Jul 12 16:04:21.596437 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:21.596411 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7558dcb54-992w4_ee1fa10f-0d01-48c3-ab56-790d581beb5e/copy-manifests/0.log" Jul 12 16:04:21.762884 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:21.762852 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-qbdk6_4039edfc-5466-49ed-807c-90f8215510d0/postgres/0.log" Jul 12 16:04:23.245243 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:23.245167 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59c7cf6567-88g9l_56cec084-7880-4945-94af-0a7eac04bd17/manager/0.log" Jul 12 16:04:24.142718 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:24.142682 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fb64c5677-89cjz_96b55013-d5e3-4886-b981-c8fc69eab194/kube-auth-proxy/0.log" Jul 12 16:04:25.578070 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:25.578027 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-6fb64c5677-89cjz_96b55013-d5e3-4886-b981-c8fc69eab194/kube-auth-proxy/0.log" Jul 12 16:04:28.242898 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:28.242863 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-4hqf6_d6f1420e-38bd-48e2-9c5d-7d34435f7f33/migrator/0.log" Jul 12 16:04:28.260752 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:28.260728 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-4hqf6_d6f1420e-38bd-48e2-9c5d-7d34435f7f33/graceful-termination/0.log" Jul 12 16:04:29.634135 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.634108 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/kube-multus-additional-cni-plugins/0.log" Jul 12 16:04:29.652848 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.652812 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/egress-router-binary-copy/0.log" Jul 12 16:04:29.672597 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.672565 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/cni-plugins/0.log" Jul 12 16:04:29.690565 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.690536 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/bond-cni-plugin/0.log" Jul 12 16:04:29.708507 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.708476 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/routeoverride-cni/0.log" Jul 12 16:04:29.726244 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.726217 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/whereabouts-cni-bincopy/0.log" Jul 12 16:04:29.744066 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.744021 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-hvncx_627256e1-05a0-4ce5-afa6-9c582f6161bf/whereabouts-cni/0.log" Jul 12 16:04:29.923016 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.922938 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-jvjfn_66a7eaab-82df-44e4-8fd2-0b0cd83648f7/kube-multus/0.log" Jul 12 16:04:29.975158 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.975122 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5x2w7_c18abc6a-0f18-4948-8c24-6bda26a89278/network-metrics-daemon/0.log" Jul 12 16:04:29.991161 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:29.991127 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-5x2w7_c18abc6a-0f18-4948-8c24-6bda26a89278/kube-rbac-proxy/0.log" Jul 12 16:04:31.029190 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.029158 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-controller/0.log" Jul 12 16:04:31.043456 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.043424 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/1.log" Jul 12 16:04:31.043456 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.043438 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/ovn-acl-logging/0.log" Jul 12 16:04:31.070881 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.070851 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/kube-rbac-proxy-node/0.log" Jul 12 16:04:31.097824 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.097794 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 16:04:31.113904 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.113874 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/northd/0.log" Jul 12 16:04:31.133032 ip-10-0-136-30 kubenswrapper[2567]: I0712 16:04:31.133003 2567 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-w686n_f57a6fd0-464d-4414-9593-5dfe4ffa93e5/nbdb/0.log"