Jul 17 18:53:08.500260 ip-10-0-133-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 17 18:53:08.500272 ip-10-0-133-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 17 18:53:08.500279 ip-10-0-133-230 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 17 18:53:08.500534 ip-10-0-133-230 systemd[1]: Failed to start Kubernetes Kubelet. Jul 17 18:53:18.535643 ip-10-0-133-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 17 18:53:18.535659 ip-10-0-133-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot a078c304ac6243f4808b8c8f74d8a595 -- Jul 17 18:55:18.233995 ip-10-0-133-230 systemd[1]: Starting Kubernetes Kubelet... Jul 17 18:55:18.613147 ip-10-0-133-230 kubenswrapper[2584]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 18:55:18.613147 ip-10-0-133-230 kubenswrapper[2584]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 17 18:55:18.613147 ip-10-0-133-230 kubenswrapper[2584]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 18:55:18.613147 ip-10-0-133-230 kubenswrapper[2584]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 17 18:55:18.613147 ip-10-0-133-230 kubenswrapper[2584]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 17 18:55:18.616309 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.616217 2584 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 17 18:55:18.622636 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622614 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.622636 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622634 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.622636 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622638 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.622636 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622656 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.622636 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622660 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622662 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622665 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622668 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622670 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622673 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622675 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622679 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622682 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622684 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622687 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622690 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622692 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622694 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622697 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622699 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622702 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622704 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622707 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622709 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.622837 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622712 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622716 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622720 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622723 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622726 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622728 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622731 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622734 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622737 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622739 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622742 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622744 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622747 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622750 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622753 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622755 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622757 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622760 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622763 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.623308 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622765 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622768 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622771 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622774 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622776 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622779 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622781 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622785 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622787 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622790 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622792 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622795 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622798 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622801 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622803 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622806 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622809 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622811 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622813 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622816 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.623830 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622818 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622821 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622824 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622826 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622828 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622831 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622836 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622839 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622843 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622846 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622851 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622854 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622858 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622862 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622866 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622870 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622872 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622875 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622877 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622880 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.624318 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622883 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622886 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622888 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622893 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622896 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622900 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622903 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622906 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622909 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622912 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622914 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622917 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622919 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622922 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622924 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.622927 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623624 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623631 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623634 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.624921 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623637 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623654 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623657 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623659 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623662 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623664 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623667 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623669 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623672 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623674 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623677 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623680 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623682 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623684 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623687 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623690 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623695 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623698 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623700 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623703 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.625379 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623705 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623707 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623710 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623713 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623715 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623719 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623723 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623726 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623728 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623731 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623734 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623738 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623740 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623743 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623746 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623748 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623751 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623754 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623757 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.625888 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623759 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623762 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623764 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623767 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623769 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623771 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623774 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623778 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623783 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623786 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623790 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623793 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623796 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623798 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623801 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623804 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623807 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623810 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623813 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.626353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623815 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623818 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623820 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623823 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623825 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623827 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623830 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623832 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623834 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623837 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623839 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623842 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623845 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623848 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623850 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623853 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623855 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623857 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623860 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623862 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.626889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623864 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623867 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623871 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623874 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623878 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623881 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623883 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623886 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623888 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623891 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623893 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623896 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623899 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623901 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623904 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623906 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623908 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.623911 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624277 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624281 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.627384 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624284 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624287 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624289 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624292 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624294 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624297 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624300 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624303 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624306 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624309 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624311 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624314 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624316 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624318 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624323 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624326 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624328 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624331 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624334 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.627897 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624336 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624339 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624341 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624344 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624346 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624349 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624351 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624354 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624356 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624358 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624361 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624363 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624366 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624368 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624371 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624373 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624378 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624381 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624384 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624387 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.628353 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624389 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624392 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624394 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624396 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624399 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624401 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624405 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624409 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624412 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624415 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624418 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624420 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624423 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624426 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624428 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624431 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624433 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624435 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624438 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.628861 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624440 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624443 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624446 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624448 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624450 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624453 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624455 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624457 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624460 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624462 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624465 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624467 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624470 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624472 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624474 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624477 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624479 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624481 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624483 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624486 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.629346 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624489 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624492 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624494 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624497 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624501 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624503 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624505 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624508 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624510 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624513 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624515 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624517 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624520 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624522 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624525 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624527 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624531 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624534 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.624536 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624603 2584 flags.go:64] FLAG: --address="0.0.0.0" Jul 17 18:55:18.629878 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624610 2584 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624616 2584 flags.go:64] FLAG: --anonymous-auth="true" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624621 2584 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624625 2584 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624628 2584 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624632 2584 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624635 2584 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624639 2584 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624655 2584 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624659 2584 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624662 2584 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624665 2584 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624669 2584 flags.go:64] FLAG: --cgroup-root="" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624671 2584 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624674 2584 flags.go:64] FLAG: --client-ca-file="" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624678 2584 flags.go:64] FLAG: --cloud-provider="external" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624681 2584 flags.go:64] FLAG: --cluster-dns="[]" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624684 2584 flags.go:64] FLAG: --cluster-domain="" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624687 2584 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624690 2584 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624693 2584 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624697 2584 flags.go:64] FLAG: --container-log-max-files="5" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624700 2584 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 17 18:55:18.630371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624703 2584 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624706 2584 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624709 2584 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624712 2584 flags.go:64] FLAG: --contention-profiling="false" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624714 2584 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624719 2584 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624722 2584 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624724 2584 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624729 2584 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624731 2584 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624734 2584 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624737 2584 flags.go:64] FLAG: --enable-load-reader="false" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624739 2584 flags.go:64] FLAG: --enable-server="true" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624742 2584 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624746 2584 flags.go:64] FLAG: --event-burst="100" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624749 2584 flags.go:64] FLAG: --event-qps="50" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624752 2584 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624755 2584 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624757 2584 flags.go:64] FLAG: --eviction-hard="" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624761 2584 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624764 2584 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624768 2584 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624771 2584 flags.go:64] FLAG: --eviction-soft="" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624774 2584 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624777 2584 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 17 18:55:18.630939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624780 2584 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624783 2584 flags.go:64] FLAG: --experimental-mounter-path="" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624786 2584 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624788 2584 flags.go:64] FLAG: --fail-swap-on="true" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624791 2584 flags.go:64] FLAG: --feature-gates="" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624794 2584 flags.go:64] FLAG: --file-check-frequency="20s" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624797 2584 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624800 2584 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624803 2584 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624806 2584 flags.go:64] FLAG: --healthz-port="10248" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624808 2584 flags.go:64] FLAG: --help="false" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624811 2584 flags.go:64] FLAG: --hostname-override="ip-10-0-133-230.ec2.internal" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624815 2584 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624818 2584 flags.go:64] FLAG: --http-check-frequency="20s" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624821 2584 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624824 2584 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624827 2584 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624830 2584 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624833 2584 flags.go:64] FLAG: --image-service-endpoint="" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624835 2584 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624838 2584 flags.go:64] FLAG: --kube-api-burst="100" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624841 2584 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624844 2584 flags.go:64] FLAG: --kube-api-qps="50" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624846 2584 flags.go:64] FLAG: --kube-reserved="" Jul 17 18:55:18.631574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624849 2584 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624852 2584 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624854 2584 flags.go:64] FLAG: --kubelet-cgroups="" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624857 2584 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624861 2584 flags.go:64] FLAG: --lock-file="" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624864 2584 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624866 2584 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624870 2584 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624875 2584 flags.go:64] FLAG: --log-json-split-stream="false" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624878 2584 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624881 2584 flags.go:64] FLAG: --log-text-split-stream="false" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624884 2584 flags.go:64] FLAG: --logging-format="text" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624886 2584 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624890 2584 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624892 2584 flags.go:64] FLAG: --manifest-url="" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624895 2584 flags.go:64] FLAG: --manifest-url-header="" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624899 2584 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624901 2584 flags.go:64] FLAG: --max-open-files="1000000" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624905 2584 flags.go:64] FLAG: --max-pods="110" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624908 2584 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624912 2584 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624954 2584 flags.go:64] FLAG: --memory-manager-policy="None" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624958 2584 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624961 2584 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624965 2584 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 17 18:55:18.632192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624968 2584 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624975 2584 flags.go:64] FLAG: --node-status-max-images="50" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624978 2584 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624981 2584 flags.go:64] FLAG: --oom-score-adj="-999" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624983 2584 flags.go:64] FLAG: --pod-cidr="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624986 2584 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4017e99d76d7b91598d3a3430cd0eba6eda8b33653addd3838fdfe3cca69c5a0" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624992 2584 flags.go:64] FLAG: --pod-manifest-path="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624994 2584 flags.go:64] FLAG: --pod-max-pids="-1" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.624997 2584 flags.go:64] FLAG: --pods-per-core="0" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625000 2584 flags.go:64] FLAG: --port="10250" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625003 2584 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625006 2584 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-01b94efa171569617" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625010 2584 flags.go:64] FLAG: --qos-reserved="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625013 2584 flags.go:64] FLAG: --read-only-port="10255" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625016 2584 flags.go:64] FLAG: --register-node="true" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625019 2584 flags.go:64] FLAG: --register-with-taints="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625022 2584 flags.go:64] FLAG: --registry-burst="10" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625026 2584 flags.go:64] FLAG: --registry-qps="5" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625029 2584 flags.go:64] FLAG: --reserved-cpus="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625032 2584 flags.go:64] FLAG: --reserved-memory="" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625035 2584 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625038 2584 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625041 2584 flags.go:64] FLAG: --rotate-certificates="false" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625043 2584 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625046 2584 flags.go:64] FLAG: --runonce="false" Jul 17 18:55:18.632794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625048 2584 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625051 2584 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625055 2584 flags.go:64] FLAG: --seccomp-default="false" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625058 2584 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625060 2584 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625063 2584 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625066 2584 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625069 2584 flags.go:64] FLAG: --storage-driver-password="root" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625071 2584 flags.go:64] FLAG: --storage-driver-secure="false" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625074 2584 flags.go:64] FLAG: --storage-driver-table="stats" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625077 2584 flags.go:64] FLAG: --storage-driver-user="root" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625080 2584 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625083 2584 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625085 2584 flags.go:64] FLAG: --system-cgroups="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625088 2584 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625093 2584 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625096 2584 flags.go:64] FLAG: --tls-cert-file="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625099 2584 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625102 2584 flags.go:64] FLAG: --tls-min-version="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625107 2584 flags.go:64] FLAG: --tls-private-key-file="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625110 2584 flags.go:64] FLAG: --topology-manager-policy="none" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625112 2584 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625115 2584 flags.go:64] FLAG: --topology-manager-scope="container" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625118 2584 flags.go:64] FLAG: --v="2" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625122 2584 flags.go:64] FLAG: --version="false" Jul 17 18:55:18.633395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625126 2584 flags.go:64] FLAG: --vmodule="" Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625130 2584 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.625133 2584 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625244 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625249 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625253 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625257 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625260 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625263 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625267 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625269 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625272 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625275 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625278 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625280 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625283 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625286 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625288 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625291 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.634016 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625293 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625296 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625299 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625301 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625304 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625306 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625308 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625312 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625314 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625317 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625319 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625321 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625324 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625326 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625329 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625332 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625334 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625337 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625339 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625342 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.634472 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625344 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625347 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625350 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625353 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625355 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625357 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625360 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625362 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625365 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625367 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625370 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625372 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625375 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625377 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625380 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625382 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625385 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625388 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625390 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625395 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.634973 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625398 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625400 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625403 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625405 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625408 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625410 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625413 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625415 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625418 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625420 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625423 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625425 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625428 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625430 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625434 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625437 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625441 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625444 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625447 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.635440 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625450 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625452 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625455 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625458 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625460 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625464 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625467 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625469 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625472 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625475 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625477 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625481 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625486 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625489 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625492 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625495 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625497 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625500 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625503 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.635959 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625505 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625508 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625511 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625513 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.625516 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.626098 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633173 2584 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633197 2584 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633216 2584 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633222 2584 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633281 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633286 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633289 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633292 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633295 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.636418 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633298 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633300 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633303 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633305 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633308 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633311 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633313 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633316 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633318 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633320 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633323 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633325 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633328 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633330 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633332 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633335 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633338 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633340 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633342 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633345 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.636889 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633347 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633350 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633352 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633354 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633365 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633368 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633371 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633373 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633376 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633378 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633381 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633383 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633386 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633388 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633392 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633396 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633399 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633402 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633405 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.637376 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633407 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633410 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633412 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633415 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633417 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633420 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633422 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633425 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633427 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633430 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633432 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633434 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633437 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633439 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633441 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633444 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633446 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633450 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633459 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633462 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.637858 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633465 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633468 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633471 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633474 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633477 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633480 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633484 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633487 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633489 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633492 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633495 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633497 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633500 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633503 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633505 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633508 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633510 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633513 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633515 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.638329 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633518 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633520 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633523 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633525 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633527 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633530 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633532 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633534 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633537 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633539 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633542 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633545 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633553 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633556 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633558 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.638796 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633561 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633566 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633716 2584 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633721 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633724 2584 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633727 2584 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633730 2584 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633732 2584 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633735 2584 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633737 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633739 2584 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633742 2584 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633744 2584 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633746 2584 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633749 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 17 18:55:18.639191 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633751 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633754 2584 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633756 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633759 2584 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633761 2584 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633764 2584 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633767 2584 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633769 2584 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633772 2584 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633774 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633776 2584 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633779 2584 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633781 2584 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633784 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633786 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633794 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633797 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633799 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633802 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 17 18:55:18.639558 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633805 2584 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633807 2584 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633810 2584 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633813 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633815 2584 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633818 2584 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633820 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633822 2584 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633825 2584 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633827 2584 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633829 2584 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633832 2584 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633834 2584 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633838 2584 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633841 2584 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633844 2584 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633846 2584 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633849 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633851 2584 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 17 18:55:18.640044 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633854 2584 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633857 2584 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633859 2584 feature_gate.go:328] unrecognized feature gate: Example Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633861 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633864 2584 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633866 2584 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633869 2584 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633871 2584 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633873 2584 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633876 2584 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633883 2584 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633886 2584 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633888 2584 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633891 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633893 2584 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633895 2584 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633898 2584 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633900 2584 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633902 2584 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633905 2584 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 17 18:55:18.640491 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633907 2584 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633911 2584 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633914 2584 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633916 2584 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633919 2584 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633921 2584 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633924 2584 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633927 2584 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633929 2584 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633932 2584 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633934 2584 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633937 2584 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633939 2584 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633942 2584 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633945 2584 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633947 2584 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633949 2584 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633952 2584 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633954 2584 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 17 18:55:18.640997 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633956 2584 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633959 2584 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633961 2584 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633964 2584 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633971 2584 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633974 2584 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633976 2584 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633978 2584 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:18.633981 2584 feature_gate.go:328] unrecognized feature gate: Example2 Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.633986 2584 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.634111 2584 server.go:962] "Client rotation is on, will bootstrap in background" Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.636084 2584 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.637855 2584 server.go:1019] "Starting client certificate rotation" Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.637959 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 18:55:18.641567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.637994 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 17 18:55:18.642033 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.640682 2584 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 18:55:18.648581 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.648562 2584 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 17 18:55:18.658214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.658194 2584 log.go:25] "Validated CRI v1 runtime API" Jul 17 18:55:18.660205 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.660193 2584 log.go:25] "Validated CRI v1 image API" Jul 17 18:55:18.661045 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.661032 2584 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 17 18:55:18.665908 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.665881 2584 fs.go:135] Filesystem UUIDs: map[6e70ee44-2259-47b2-9f73-3ae1a58ddcee:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 bd5145bb-b4f6-4a0f-bb90-c95fc1d8aed4:/dev/nvme0n1p3] Jul 17 18:55:18.665977 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.665908 2584 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 17 18:55:18.669289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.669265 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 18:55:18.671344 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.671227 2584 manager.go:216] Machine: {Timestamp:2026-07-17 18:55:18.669926865 +0000 UTC m=+0.321307311 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3133273 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e5ba938bbe197eed4ba83c4b6c4f0 SystemUUID:ec2e5ba9-38bb-e197-eed4-ba83c4b6c4f0 BootID:a078c304-ac62-43f4-808b-8c8f74d8a595 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a4:9d:42:b5:17 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a4:9d:42:b5:17 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:9a:cc:bd:9f:8f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 17 18:55:18.671344 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.671337 2584 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 17 18:55:18.671501 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.671451 2584 manager.go:232] Version: {KernelVersion:5.14.0-570.128.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260715-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 17 18:55:18.675238 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.675217 2584 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 17 18:55:18.675374 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.675241 2584 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-133-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 17 18:55:18.676479 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.676469 2584 topology_manager.go:138] "Creating topology manager with none policy" Jul 17 18:55:18.676513 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.676480 2584 container_manager_linux.go:309] "Creating device plugin manager" Jul 17 18:55:18.676513 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.676491 2584 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 18:55:18.677216 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.677206 2584 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 17 18:55:18.678260 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.678250 2584 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 17 18:55:18.680776 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680765 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 17 18:55:18.680881 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680871 2584 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 17 18:55:18.680917 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680901 2584 kubelet.go:486] "Attempting to sync node with API server" Jul 17 18:55:18.680917 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680911 2584 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 17 18:55:18.680983 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680921 2584 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 17 18:55:18.680983 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680929 2584 kubelet.go:398] "Adding apiserver pod source" Jul 17 18:55:18.680983 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.680937 2584 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 17 18:55:18.681301 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.681267 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 18:55:18.681301 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.681284 2584 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 17 18:55:18.683906 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.683891 2584 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 17 18:55:18.685512 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.685495 2584 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 17 18:55:18.685585 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.685534 2584 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 17 18:55:18.686813 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686800 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686819 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686829 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686839 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686851 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686860 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686872 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 17 18:55:18.686893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686888 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 17 18:55:18.687120 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686898 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 17 18:55:18.687120 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686907 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 17 18:55:18.687120 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686924 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 17 18:55:18.687120 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.686932 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 17 18:55:18.688258 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.688245 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 17 18:55:18.688315 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.688261 2584 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 17 18:55:18.690986 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.690949 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 17 18:55:18.690986 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.690968 2584 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-133-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 17 18:55:18.691125 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.691075 2584 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-133-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 17 18:55:18.692075 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.692059 2584 server.go:1268] "Started kubelet" Jul 17 18:55:18.692204 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.692179 2584 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 17 18:55:18.692251 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.692172 2584 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 17 18:55:18.696211 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.692232 2584 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 17 18:55:18.697085 ip-10-0-133-230 systemd[1]: Started Kubernetes Kubelet. Jul 17 18:55:18.697839 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.697808 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzpx6" Jul 17 18:55:18.698790 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.698774 2584 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 17 18:55:18.701319 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.701303 2584 server.go:310] "Adding debug handlers to kubelet server" Jul 17 18:55:18.703412 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.703395 2584 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 17 18:55:18.704818 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.703885 2584 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-133-230.ec2.internal.18c32899de35e77d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-133-230.ec2.internal,UID:ip-10-0-133-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-133-230.ec2.internal,},FirstTimestamp:2026-07-17 18:55:18.692030333 +0000 UTC m=+0.343410778,LastTimestamp:2026-07-17 18:55:18.692030333 +0000 UTC m=+0.343410778,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-133-230.ec2.internal,}" Jul 17 18:55:18.705067 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.705052 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mzpx6" Jul 17 18:55:18.705622 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.705608 2584 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 17 18:55:18.705695 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.705627 2584 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 17 18:55:18.705938 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.705814 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:18.706179 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706160 2584 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 17 18:55:18.706179 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706157 2584 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 17 18:55:18.706303 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706189 2584 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 17 18:55:18.706359 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706313 2584 reconstruct.go:97] "Volume reconstruction finished" Jul 17 18:55:18.706359 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706320 2584 reconciler.go:29] "Reconciler: start to sync state" Jul 17 18:55:18.706461 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706420 2584 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 17 18:55:18.706461 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706434 2584 factory.go:55] Registering systemd factory Jul 17 18:55:18.706461 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706442 2584 factory.go:223] Registration of the systemd container factory successfully Jul 17 18:55:18.706744 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706732 2584 factory.go:153] Registering CRI-O factory Jul 17 18:55:18.706744 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706745 2584 factory.go:223] Registration of the crio container factory successfully Jul 17 18:55:18.706850 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706765 2584 factory.go:103] Registering Raw factory Jul 17 18:55:18.706850 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.706775 2584 manager.go:1196] Started watching for new ooms in manager Jul 17 18:55:18.707310 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.707297 2584 manager.go:349] Starting recovery of all containers Jul 17 18:55:18.713797 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.713774 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:18.717546 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.717397 2584 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-133-230.ec2.internal\" not found" node="ip-10-0-133-230.ec2.internal" Jul 17 18:55:18.717546 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.717478 2584 manager.go:354] Recovery completed Jul 17 18:55:18.722726 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.722714 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:18.724806 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.724776 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:18.724806 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.724805 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:18.724884 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.724821 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:18.725360 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.725346 2584 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 17 18:55:18.725438 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.725359 2584 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 17 18:55:18.725438 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.725380 2584 state_mem.go:36] "Initialized new in-memory state store" Jul 17 18:55:18.725438 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.725350 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:18.726085 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726072 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:18.726140 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726095 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:18.726140 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726108 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:18.726939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726926 2584 policy_none.go:49] "None policy: Start" Jul 17 18:55:18.726997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726945 2584 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 17 18:55:18.726997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.726958 2584 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 17 18:55:18.727853 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.727840 2584 policy_none.go:47] "Start" Jul 17 18:55:18.767274 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767256 2584 manager.go:337] "Starting Device Plugin manager" Jul 17 18:55:18.767349 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.767288 2584 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 17 18:55:18.767349 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767297 2584 server.go:87] "Starting device plugin registration server" Jul 17 18:55:18.767560 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767547 2584 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 17 18:55:18.767669 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767561 2584 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 17 18:55:18.767669 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767638 2584 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 17 18:55:18.767761 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767729 2584 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 17 18:55:18.767761 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.767739 2584 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 17 18:55:18.768273 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.768256 2584 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 17 18:55:18.768333 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.768302 2584 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:18.868187 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.868120 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:18.869226 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.869049 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:18.869226 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.869080 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:18.869226 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.869096 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:18.869226 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.869123 2584 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-133-230.ec2.internal" Jul 17 18:55:18.877506 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.877494 2584 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-133-230.ec2.internal" Jul 17 18:55:18.877556 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.877513 2584 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-133-230.ec2.internal\": node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:18.894832 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.894810 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:18.896798 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.896770 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 17 18:55:18.898132 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.898119 2584 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 17 18:55:18.898204 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.898156 2584 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 17 18:55:18.898204 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.898177 2584 kubelet.go:2448] "Starting kubelet main sync loop" Jul 17 18:55:18.898294 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.898211 2584 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 17 18:55:18.900884 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.900867 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:18.995802 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:18.995758 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:18.998922 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.998900 2584 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal"] Jul 17 18:55:18.998982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.998973 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:18.999961 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.999945 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:19.000069 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.999975 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:19.000069 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:18.999987 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:19.001284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001271 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:19.001413 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001397 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.001472 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001434 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:19.001980 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001963 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:19.002060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001982 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:19.002060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.001991 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:19.002060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.002002 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:19.002060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.002005 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:19.002060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.002019 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:19.003116 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.003099 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.003182 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.003128 2584 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 17 18:55:19.003803 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.003789 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientMemory" Jul 17 18:55:19.003862 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.003814 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasNoDiskPressure" Jul 17 18:55:19.003862 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.003827 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeHasSufficientPID" Jul 17 18:55:19.007693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.007674 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50351eb0dda3c09fbc3ee35a82fae765-config\") pod \"kube-apiserver-proxy-ip-10-0-133-230.ec2.internal\" (UID: \"50351eb0dda3c09fbc3ee35a82fae765\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.007773 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.007698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.007773 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.007716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.018169 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.018150 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-230.ec2.internal\" not found" node="ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.022504 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.022488 2584 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-133-230.ec2.internal\" not found" node="ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.096232 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.096210 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.108531 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108512 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.108584 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108540 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.108584 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108556 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50351eb0dda3c09fbc3ee35a82fae765-config\") pod \"kube-apiserver-proxy-ip-10-0-133-230.ec2.internal\" (UID: \"50351eb0dda3c09fbc3ee35a82fae765\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.108703 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108612 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.108703 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108612 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/50351eb0dda3c09fbc3ee35a82fae765-config\") pod \"kube-apiserver-proxy-ip-10-0-133-230.ec2.internal\" (UID: \"50351eb0dda3c09fbc3ee35a82fae765\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.108703 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.108664 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/5e1580e7866c089718b70d4daea63a50-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal\" (UID: \"5e1580e7866c089718b70d4daea63a50\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.196395 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.196319 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.296847 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.296816 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.321367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.321344 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.326361 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.326054 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.397581 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.397548 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.498471 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.498446 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.605117 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.605072 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.636745 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.636718 2584 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 17 18:55:19.637234 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.636871 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 18:55:19.637234 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.636909 2584 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 17 18:55:19.705358 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:19.705333 2584 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-133-230.ec2.internal\" not found" Jul 17 18:55:19.710017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.709981 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-16 18:50:19 +0000 UTC" deadline="2028-04-09 07:12:17.612441754 +0000 UTC" Jul 17 18:55:19.710017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.710018 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="15156h16m57.902426486s" Jul 17 18:55:19.795929 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.795859 2584 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:19.805881 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.805848 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.817635 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.817614 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 18:55:19.819617 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.819598 2584 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" Jul 17 18:55:19.833689 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.833669 2584 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 17 18:55:19.852414 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:19.852396 2584 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:20.018873 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:20.018840 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50351eb0dda3c09fbc3ee35a82fae765.slice/crio-59053330f21960e579adb55834c468605b967d209a44474bbdb6cffe9d8c7ca2 WatchSource:0}: Error finding container 59053330f21960e579adb55834c468605b967d209a44474bbdb6cffe9d8c7ca2: Status 404 returned error can't find the container with id 59053330f21960e579adb55834c468605b967d209a44474bbdb6cffe9d8c7ca2 Jul 17 18:55:20.019331 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:20.019303 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e1580e7866c089718b70d4daea63a50.slice/crio-f76a214b8ae3c26a86abc6a6eb75d51fed368cb46fffb474abed35256a5095a0 WatchSource:0}: Error finding container f76a214b8ae3c26a86abc6a6eb75d51fed368cb46fffb474abed35256a5095a0: Status 404 returned error can't find the container with id f76a214b8ae3c26a86abc6a6eb75d51fed368cb46fffb474abed35256a5095a0 Jul 17 18:55:20.026229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.026201 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 18:55:20.684284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.684228 2584 apiserver.go:52] "Watching apiserver" Jul 17 18:55:20.693435 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.693410 2584 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 17 18:55:20.695506 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.695480 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal","openshift-dns/node-resolver-h9bzt","openshift-image-registry/node-ca-jhjxm","openshift-multus/network-metrics-daemon-lc7dh","openshift-network-operator/iptables-alerter-fr466","openshift-ovn-kubernetes/ovnkube-node-hhckh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn","openshift-cluster-node-tuning-operator/tuned-s5x6p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-vn44d","openshift-multus/multus-rpcgw","openshift-network-diagnostics/network-check-target-6jwkm","kube-system/global-pull-secret-syncer-f67qc","kube-system/konnectivity-agent-8q9k6"] Jul 17 18:55:20.699586 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.699567 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.702398 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.702378 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.702516 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.702467 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:20.702765 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.702743 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ckbw4\"" Jul 17 18:55:20.702834 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.702751 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.702834 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.702788 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 17 18:55:20.702919 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.702753 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.704688 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.704671 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.706790 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.706765 2584 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 17 18:55:20.707127 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.707108 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.707211 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.707179 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:20.707677 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.707656 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-nqqts\"" Jul 17 18:55:20.707786 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.707701 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 17 18:55:20.707786 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.707710 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.707786 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.707739 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.709464 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.709450 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.712752 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.712251 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.712752 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.712250 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.712752 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.712283 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.712752 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.712291 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-p2m2d\"" Jul 17 18:55:20.713078 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.712757 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 17 18:55:20.714982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.714963 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 17 18:55:20.715083 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715013 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 17 18:55:20.715306 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715084 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.715306 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715222 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.715583 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715566 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 17 18:55:20.715651 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715636 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 17 18:55:20.715735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715566 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.715735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.715667 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dfwq7\"" Jul 17 18:55:20.717897 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.717880 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.717994 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.717960 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-cn88j\"" Jul 17 18:55:20.718280 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.718077 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 17 18:55:20.718396 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.718376 2584 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 17 18:55:20.718483 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.718403 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 17 18:55:20.719962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.719921 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-device-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.719962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.719951 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-serviceca\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.720122 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.719971 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q7wg9\" (UniqueName: \"kubernetes.io/projected/1b173869-5333-421b-8a0f-f8dfc9a72faf-kube-api-access-q7wg9\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.720122 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720024 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-registration-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720122 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720063 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-sys-fs\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720122 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720091 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-dbus\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720147 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dzlv\" (UniqueName: \"kubernetes.io/projected/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kube-api-access-4dzlv\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720164 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-kubelet-config\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720186 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-host\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720212 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720237 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720290 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720290 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54fsx\" (UniqueName: \"kubernetes.io/projected/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-kube-api-access-54fsx\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.720391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720381 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.720827 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720407 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-socket-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.720933 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720912 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wjcjh\"" Jul 17 18:55:20.721027 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720912 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.721191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.720920 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.724482 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.724462 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-282dl\"" Jul 17 18:55:20.724597 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.724514 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.725017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.724985 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.725801 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.725782 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.727451 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.727424 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.727776 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.727721 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-7994l\"" Jul 17 18:55:20.728902 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.728638 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 17 18:55:20.728902 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.728638 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 17 18:55:20.728902 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.728710 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 17 18:55:20.729097 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.728910 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 17 18:55:20.729097 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.728920 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 17 18:55:20.730128 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.730107 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:20.730240 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.730173 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:20.730515 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.730493 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 17 18:55:20.730643 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.730624 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-sq5d2\"" Jul 17 18:55:20.746698 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.746674 2584 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-v92mz" Jul 17 18:55:20.759032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.758877 2584 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-v92mz" Jul 17 18:55:20.807639 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.807611 2584 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 17 18:55:20.821123 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821096 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-hostroot\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821133 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-kubelet-config\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-host\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821189 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-kubelet\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821239 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250faa54-3627-40ca-b09a-3587e23e29cc-host-slash\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821255 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-kubernetes\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.821289 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821274 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-kubelet-config\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821279 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-host\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821282 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821334 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0a06bcd-e178-4db3-891d-ec3d90901d1e-agent-certs\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.821382 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821363 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbcb13f3-7841-445c-a433-5234b674aeca-hosts-file\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821425 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.821470 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:21.321435172 +0000 UTC m=+2.972815607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q7wg9\" (UniqueName: \"kubernetes.io/projected/1b173869-5333-421b-8a0f-f8dfc9a72faf-kube-api-access-q7wg9\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821550 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-conf\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821577 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821604 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-k8s-cni-cncf-io\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.821625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821678 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-sys-fs\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821714 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-sys\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821769 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821776 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-sys-fs\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821812 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hrbrm\" (UniqueName: \"kubernetes.io/projected/250faa54-3627-40ca-b09a-3587e23e29cc-kube-api-access-hrbrm\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821861 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-etc-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821903 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-var-lib-kubelet\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.821978 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-cnibin\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822045 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-systemd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822051 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-etc-selinux\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822066 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovn-node-metrics-cert\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822086 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822109 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-host\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822129 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.822167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822146 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-var-lib-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822194 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kubelet-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822224 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-env-overrides\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822256 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-socket-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822284 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-device-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822308 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-modprobe-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822343 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822367 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-cnibin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822391 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-multus-certs\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822400 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-device-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822413 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-run\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822453 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-socket-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-bin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822494 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822523 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-script-lib\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822547 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.822887 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822571 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-tmp\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-os-release\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822629 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-netns\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822656 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-systemd-units\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-bin\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822817 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysconfig\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-54fsx\" (UniqueName: \"kubernetes.io/projected/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-kube-api-access-54fsx\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb13f3-7841-445c-a433-5234b674aeca-tmp-dir\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822896 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-node-log\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822920 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-lib-modules\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822969 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-os-release\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.822994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-cni-binary-copy\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823020 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsvtx\" (UniqueName: \"kubernetes.io/projected/aa36ca8f-467b-4d37-8eae-125febadac44-kube-api-access-hsvtx\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823062 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-kubelet\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823087 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-registration-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823113 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8d44\" (UniqueName: \"kubernetes.io/projected/dbcb13f3-7841-445c-a433-5234b674aeca-kube-api-access-x8d44\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.823600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823141 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-system-cni-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823168 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pzgs\" (UniqueName: \"kubernetes.io/projected/f8f76668-6cf7-48a3-88dc-998738fa44c4-kube-api-access-7pzgs\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823187 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-registration-dir\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823192 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-multus\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823265 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-conf-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823285 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-ovn\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823301 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-config\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823326 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dzlv\" (UniqueName: \"kubernetes.io/projected/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kube-api-access-4dzlv\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823358 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-multus-daemon-config\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823390 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-slash\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823496 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-netns\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823518 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-log-socket\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823540 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-netd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823563 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v52h\" (UniqueName: \"kubernetes.io/projected/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-kube-api-access-8v52h\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823599 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823632 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0a06bcd-e178-4db3-891d-ec3d90901d1e-konnectivity-ca\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.824231 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823672 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-system-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823711 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-socket-dir-parent\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.823734 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823741 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-etc-kubernetes\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.823793 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:21.32377484 +0000 UTC m=+2.975155274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823813 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-serviceca\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823830 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ft9jc\" (UniqueName: \"kubernetes.io/projected/04120778-d92c-40bd-8d4e-e300c09c44fe-kube-api-access-ft9jc\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250faa54-3627-40ca-b09a-3587e23e29cc-iptables-alerter-script\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823893 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823920 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-dbus\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-systemd\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.823966 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-tuned\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.824067 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-dbus\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:20.824880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.824342 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-serviceca\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.829757 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.829734 2584 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 17 18:55:20.833575 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.833553 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q7wg9\" (UniqueName: \"kubernetes.io/projected/1b173869-5333-421b-8a0f-f8dfc9a72faf-kube-api-access-q7wg9\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:20.833670 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.833575 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dzlv\" (UniqueName: \"kubernetes.io/projected/0b57b1f4-92c2-4f9c-942c-d7a8375324dd-kube-api-access-4dzlv\") pod \"aws-ebs-csi-driver-node-x9tqn\" (UID: \"0b57b1f4-92c2-4f9c-942c-d7a8375324dd\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:20.833670 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.833575 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-54fsx\" (UniqueName: \"kubernetes.io/projected/792eb6c8-4a63-4882-8985-10b8bcf3a1b9-kube-api-access-54fsx\") pod \"node-ca-jhjxm\" (UID: \"792eb6c8-4a63-4882-8985-10b8bcf3a1b9\") " pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:20.903626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.903565 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" event={"ID":"50351eb0dda3c09fbc3ee35a82fae765","Type":"ContainerStarted","Data":"59053330f21960e579adb55834c468605b967d209a44474bbdb6cffe9d8c7ca2"} Jul 17 18:55:20.906532 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.906442 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" event={"ID":"5e1580e7866c089718b70d4daea63a50","Type":"ContainerStarted","Data":"f76a214b8ae3c26a86abc6a6eb75d51fed368cb46fffb474abed35256a5095a0"} Jul 17 18:55:20.924252 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924215 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-lib-modules\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924264 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924292 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-os-release\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924314 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-cni-binary-copy\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924340 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsvtx\" (UniqueName: \"kubernetes.io/projected/aa36ca8f-467b-4d37-8eae-125febadac44-kube-api-access-hsvtx\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924363 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-kubelet\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.924407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924387 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8d44\" (UniqueName: \"kubernetes.io/projected/dbcb13f3-7841-445c-a433-5234b674aeca-kube-api-access-x8d44\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-system-cni-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924410 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-os-release\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924429 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-lib-modules\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924437 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pzgs\" (UniqueName: \"kubernetes.io/projected/f8f76668-6cf7-48a3-88dc-998738fa44c4-kube-api-access-7pzgs\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924481 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-kubelet\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-multus\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924519 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-multus\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924529 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-conf-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924557 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-ovn\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924581 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-config\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924607 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-multus-daemon-config\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924632 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-slash\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.924704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924697 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-system-cni-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924746 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-slash\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924767 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-ovn\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924775 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-netns\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924802 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-log-socket\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-netd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924851 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v52h\" (UniqueName: \"kubernetes.io/projected/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-kube-api-access-8v52h\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924891 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0a06bcd-e178-4db3-891d-ec3d90901d1e-konnectivity-ca\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924917 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-system-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924941 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-socket-dir-parent\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924973 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-etc-kubernetes\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925002 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ft9jc\" (UniqueName: \"kubernetes.io/projected/04120778-d92c-40bd-8d4e-e300c09c44fe-kube-api-access-ft9jc\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250faa54-3627-40ca-b09a-3587e23e29cc-iptables-alerter-script\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925089 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-systemd\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925113 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-tuned\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925135 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-hostroot\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-kubelet\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.925229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925190 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250faa54-3627-40ca-b09a-3587e23e29cc-host-slash\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924940 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925263 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-netns\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925276 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-etc-kubernetes\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925318 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-netd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-log-socket\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.924974 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-conf-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925397 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-system-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-kubernetes\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925470 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0a06bcd-e178-4db3-891d-ec3d90901d1e-agent-certs\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925489 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbcb13f3-7841-445c-a433-5234b674aeca-hosts-file\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925506 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-conf\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925587 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925617 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-k8s-cni-cncf-io\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925644 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925678 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-sys\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.926202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925700 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/dbcb13f3-7841-445c-a433-5234b674aeca-hosts-file\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925713 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925741 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-cni-binary-copy\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925763 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hrbrm\" (UniqueName: \"kubernetes.io/projected/250faa54-3627-40ca-b09a-3587e23e29cc-kube-api-access-hrbrm\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925779 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/aa36ca8f-467b-4d37-8eae-125febadac44-multus-daemon-config\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925791 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-kubernetes\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925791 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-etc-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-var-lib-kubelet\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925847 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-systemd\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925858 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-etc-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925870 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-cnibin\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925900 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-systemd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925920 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-hostroot\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovn-node-metrics-cert\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925953 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-host\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/250faa54-3627-40ca-b09a-3587e23e29cc-host-slash\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.927245 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926034 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-var-lib-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926058 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-env-overrides\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.925651 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-config\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-modprobe-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926109 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926140 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-cnibin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926167 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-multus-certs\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926197 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-run\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926202 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-tuning-conf-dir\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926222 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-bin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926273 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/250faa54-3627-40ca-b09a-3587e23e29cc-iptables-alerter-script\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926279 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-run-systemd\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926330 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926003 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-conf\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926397 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-host\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-modprobe-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926456 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-cni-dir\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.928212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926476 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-var-lib-openvswitch\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926051 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-multus-socket-dir-parent\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926524 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d0a06bcd-e178-4db3-891d-ec3d90901d1e-konnectivity-ca\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926530 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-multus-certs\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926113 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-k8s-cni-cncf-io\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926599 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-cnibin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926825 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-cnibin\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926861 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-cni-bin\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926893 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-var-lib-kubelet\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926920 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-run\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926944 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-var-lib-kubelet\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926961 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926986 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-run-ovn-kubernetes\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.926994 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-script-lib\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927006 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-env-overrides\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927044 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-cni-binary-copy\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927050 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927089 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-sys\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.929145 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927106 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-tmp\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927145 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-os-release\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927164 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysctl-d\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927172 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-netns\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927198 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-systemd-units\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927223 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-bin\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927249 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysconfig\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927255 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8f76668-6cf7-48a3-88dc-998738fa44c4-os-release\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927276 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb13f3-7841-445c-a433-5234b674aeca-tmp-dir\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927303 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-node-log\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927326 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/aa36ca8f-467b-4d37-8eae-125febadac44-host-run-netns\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927384 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-node-log\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927383 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8f76668-6cf7-48a3-88dc-998738fa44c4-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927384 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-host-cni-bin\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927430 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-systemd-units\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927457 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-sysconfig\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927496 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovnkube-script-lib\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.927701 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dbcb13f3-7841-445c-a433-5234b674aeca-tmp-dir\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.930015 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.929251 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-ovn-node-metrics-cert\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.930585 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.929896 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d0a06bcd-e178-4db3-891d-ec3d90901d1e-agent-certs\") pod \"konnectivity-agent-8q9k6\" (UID: \"d0a06bcd-e178-4db3-891d-ec3d90901d1e\") " pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:20.930585 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.929991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-etc-tuned\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.930585 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.930161 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/04120778-d92c-40bd-8d4e-e300c09c44fe-tmp\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.933638 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.933212 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:20.933638 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.933236 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:20.933638 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.933249 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:20.933638 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:20.933337 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:21.433317835 +0000 UTC m=+3.084698299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:20.935827 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.935773 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ft9jc\" (UniqueName: \"kubernetes.io/projected/04120778-d92c-40bd-8d4e-e300c09c44fe-kube-api-access-ft9jc\") pod \"tuned-s5x6p\" (UID: \"04120778-d92c-40bd-8d4e-e300c09c44fe\") " pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:20.936091 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.936071 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsvtx\" (UniqueName: \"kubernetes.io/projected/aa36ca8f-467b-4d37-8eae-125febadac44-kube-api-access-hsvtx\") pod \"multus-rpcgw\" (UID: \"aa36ca8f-467b-4d37-8eae-125febadac44\") " pod="openshift-multus/multus-rpcgw" Jul 17 18:55:20.936591 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.936564 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pzgs\" (UniqueName: \"kubernetes.io/projected/f8f76668-6cf7-48a3-88dc-998738fa44c4-kube-api-access-7pzgs\") pod \"multus-additional-cni-plugins-vn44d\" (UID: \"f8f76668-6cf7-48a3-88dc-998738fa44c4\") " pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:20.937467 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.937437 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v52h\" (UniqueName: \"kubernetes.io/projected/e52695ee-b0ac-4f2d-9c0e-4ee432280f05-kube-api-access-8v52h\") pod \"ovnkube-node-hhckh\" (UID: \"e52695ee-b0ac-4f2d-9c0e-4ee432280f05\") " pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:20.937467 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.937459 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8d44\" (UniqueName: \"kubernetes.io/projected/dbcb13f3-7841-445c-a433-5234b674aeca-kube-api-access-x8d44\") pod \"node-resolver-h9bzt\" (UID: \"dbcb13f3-7841-445c-a433-5234b674aeca\") " pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:20.938956 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:20.938933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hrbrm\" (UniqueName: \"kubernetes.io/projected/250faa54-3627-40ca-b09a-3587e23e29cc-kube-api-access-hrbrm\") pod \"iptables-alerter-fr466\" (UID: \"250faa54-3627-40ca-b09a-3587e23e29cc\") " pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:21.009196 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.009159 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" Jul 17 18:55:21.016017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.015824 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-jhjxm" Jul 17 18:55:21.021856 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:21.021825 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b57b1f4_92c2_4f9c_942c_d7a8375324dd.slice/crio-eef7c74b819f5847e5a4edc5d8ec0233e1b7bc40f82f956451ae0ae5dba29f25 WatchSource:0}: Error finding container eef7c74b819f5847e5a4edc5d8ec0233e1b7bc40f82f956451ae0ae5dba29f25: Status 404 returned error can't find the container with id eef7c74b819f5847e5a4edc5d8ec0233e1b7bc40f82f956451ae0ae5dba29f25 Jul 17 18:55:21.026539 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.026478 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-fr466" Jul 17 18:55:21.033882 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.033859 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:21.041440 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.041420 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:21.049123 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.049102 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" Jul 17 18:55:21.055285 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.055266 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-h9bzt" Jul 17 18:55:21.063908 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.063887 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-vn44d" Jul 17 18:55:21.066040 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:21.065985 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04120778_d92c_40bd_8d4e_e300c09c44fe.slice/crio-4c9d04bcbd1f7b044ffb8fa491f8bfbf8728f6f83a2a627c53f2dc7b90cf2b3c WatchSource:0}: Error finding container 4c9d04bcbd1f7b044ffb8fa491f8bfbf8728f6f83a2a627c53f2dc7b90cf2b3c: Status 404 returned error can't find the container with id 4c9d04bcbd1f7b044ffb8fa491f8bfbf8728f6f83a2a627c53f2dc7b90cf2b3c Jul 17 18:55:21.067965 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:21.067932 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0a06bcd_e178_4db3_891d_ec3d90901d1e.slice/crio-7a8bafeaab6581b8eae1b2ba2774fdb20229e6f1f4eea2af6bdf431e1058718a WatchSource:0}: Error finding container 7a8bafeaab6581b8eae1b2ba2774fdb20229e6f1f4eea2af6bdf431e1058718a: Status 404 returned error can't find the container with id 7a8bafeaab6581b8eae1b2ba2774fdb20229e6f1f4eea2af6bdf431e1058718a Jul 17 18:55:21.068584 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.068561 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-rpcgw" Jul 17 18:55:21.071309 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:21.071256 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddbcb13f3_7841_445c_a433_5234b674aeca.slice/crio-72540befb639602acc75d8ae6042da370da6a0fa2d186b49ce4b9bae1b2e3bc4 WatchSource:0}: Error finding container 72540befb639602acc75d8ae6042da370da6a0fa2d186b49ce4b9bae1b2e3bc4: Status 404 returned error can't find the container with id 72540befb639602acc75d8ae6042da370da6a0fa2d186b49ce4b9bae1b2e3bc4 Jul 17 18:55:21.080174 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:55:21.080153 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8f76668_6cf7_48a3_88dc_998738fa44c4.slice/crio-0e4402e84b0b6b8b75b80a9076bf9844eae065d1b363e81eb9649ff716f36f02 WatchSource:0}: Error finding container 0e4402e84b0b6b8b75b80a9076bf9844eae065d1b363e81eb9649ff716f36f02: Status 404 returned error can't find the container with id 0e4402e84b0b6b8b75b80a9076bf9844eae065d1b363e81eb9649ff716f36f02 Jul 17 18:55:21.117182 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.117154 2584 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:21.145266 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.145226 2584 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 17 18:55:21.330320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.330237 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:21.330479 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.330333 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:21.330479 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.330379 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:21.330479 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.330440 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:21.330479 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.330464 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:22.330442632 +0000 UTC m=+3.981823079 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:21.330645 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.330493 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:22.330475968 +0000 UTC m=+3.981856420 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:21.532347 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.532312 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:21.532525 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.532501 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:21.532597 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.532532 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:21.532597 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.532547 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:21.532721 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.532621 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:22.532592116 +0000 UTC m=+4.183972569 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:21.764123 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.764078 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 18:50:21 +0000 UTC" deadline="2028-04-13 22:49:05.132551238 +0000 UTC" Jul 17 18:55:21.764123 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.764115 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15267h53m43.368440244s" Jul 17 18:55:21.899241 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.899213 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:21.899391 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:21.899351 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:21.910478 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.910444 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerStarted","Data":"0e4402e84b0b6b8b75b80a9076bf9844eae065d1b363e81eb9649ff716f36f02"} Jul 17 18:55:21.912043 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.911988 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" event={"ID":"04120778-d92c-40bd-8d4e-e300c09c44fe","Type":"ContainerStarted","Data":"4c9d04bcbd1f7b044ffb8fa491f8bfbf8728f6f83a2a627c53f2dc7b90cf2b3c"} Jul 17 18:55:21.914003 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.913906 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"1b5af9b0803e0dabbe2c36ade953afcff4a84c001e11dc3c2ea94745ee607f19"} Jul 17 18:55:21.915445 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.915414 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" event={"ID":"0b57b1f4-92c2-4f9c-942c-d7a8375324dd","Type":"ContainerStarted","Data":"eef7c74b819f5847e5a4edc5d8ec0233e1b7bc40f82f956451ae0ae5dba29f25"} Jul 17 18:55:21.917367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.917328 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rpcgw" event={"ID":"aa36ca8f-467b-4d37-8eae-125febadac44","Type":"ContainerStarted","Data":"69708f1b7cc4b675c45e5aa7cd4bd7cf69120e90c0b6a5d47a784ad6f68ed39f"} Jul 17 18:55:21.919098 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.919068 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h9bzt" event={"ID":"dbcb13f3-7841-445c-a433-5234b674aeca","Type":"ContainerStarted","Data":"72540befb639602acc75d8ae6042da370da6a0fa2d186b49ce4b9bae1b2e3bc4"} Jul 17 18:55:21.920609 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.920582 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8q9k6" event={"ID":"d0a06bcd-e178-4db3-891d-ec3d90901d1e","Type":"ContainerStarted","Data":"7a8bafeaab6581b8eae1b2ba2774fdb20229e6f1f4eea2af6bdf431e1058718a"} Jul 17 18:55:21.921992 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.921965 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fr466" event={"ID":"250faa54-3627-40ca-b09a-3587e23e29cc","Type":"ContainerStarted","Data":"17ab8a8e2bb89335f4e1a9444492ca6e5f735e95c1f66cc28b1c908b728e1c4e"} Jul 17 18:55:21.923391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:21.923370 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jhjxm" event={"ID":"792eb6c8-4a63-4882-8985-10b8bcf3a1b9","Type":"ContainerStarted","Data":"540e27268e5ef1380c42c34bf3abb19652723eb1b2d74a6799a7c3a6827d04d7"} Jul 17 18:55:22.343115 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.343068 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:22.343330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.343178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:22.343330 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.343321 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:22.343462 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.343387 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:24.343366108 +0000 UTC m=+5.994746547 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:22.343462 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.343454 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:22.343590 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.343487 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:24.343475843 +0000 UTC m=+5.994856286 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:22.545398 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.545355 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:22.545578 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.545550 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:22.545578 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.545569 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:22.545743 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.545583 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:22.545743 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.545644 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:24.545623255 +0000 UTC m=+6.197003693 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:22.765355 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.765242 2584 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-16 18:50:21 +0000 UTC" deadline="2028-04-06 01:30:08.534468758 +0000 UTC" Jul 17 18:55:22.765355 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.765305 2584 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15078h34m45.769168637s" Jul 17 18:55:22.901586 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.901553 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:22.901774 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.901704 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:22.902159 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:22.902134 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:22.902254 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:22.902230 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:23.899365 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:23.898810 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:23.899365 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:23.898961 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:24.360777 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:24.360675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:24.360777 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:24.360738 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:24.361024 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.360868 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:24.361024 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.360868 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:24.361024 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.360926 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:28.360906955 +0000 UTC m=+10.012287391 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:24.361024 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.360944 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:28.360934701 +0000 UTC m=+10.012315135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:24.562802 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:24.562768 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:24.563524 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.562971 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:24.563524 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.562997 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:24.563524 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.563012 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:24.563524 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.563068 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:28.563048937 +0000 UTC m=+10.214429377 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:24.899336 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:24.898747 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:24.899336 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.898899 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:24.900150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:24.899980 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:24.900150 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:24.900104 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:25.899475 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:25.898889 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:25.899475 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:25.899031 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:26.898764 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:26.898719 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:26.898942 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:26.898872 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:26.900719 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:26.900691 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:26.901131 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:26.900827 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:27.899288 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:27.899245 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:27.899500 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:27.899398 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:28.397158 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:28.397220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.397377 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.397440 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:36.397420582 +0000 UTC m=+18.048801018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.397845 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:28.397992 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.397899 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:36.397882667 +0000 UTC m=+18.049263104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:28.598872 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:28.598834 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:28.599041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.598998 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:28.599041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.599016 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:28.599041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.599028 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:28.599213 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.599074 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:36.599061847 +0000 UTC m=+18.250442282 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:28.899916 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:28.899665 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:28.900070 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:28.899719 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:28.900070 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.900034 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:28.900196 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:28.900088 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:29.898896 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:29.898865 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:29.899329 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:29.898989 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:30.899435 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:30.899403 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:30.899944 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:30.899542 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:30.900055 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:30.900025 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:30.900161 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:30.900141 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:31.898672 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:31.898618 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:31.899003 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:31.898966 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:32.899198 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:32.899126 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:32.899529 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:32.899243 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:32.899529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:32.899299 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:32.899529 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:32.899403 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:33.898781 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:33.898741 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:33.898951 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:33.898867 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:34.899343 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:34.899299 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:34.899756 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:34.899309 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:34.899756 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:34.899453 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:34.899756 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:34.899489 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:35.899027 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:35.898987 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:35.899233 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:35.899105 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:36.459638 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:36.459600 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:36.460075 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:36.459666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:36.460075 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.459767 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:36.460075 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.459773 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:36.460075 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.459838 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:52.459821145 +0000 UTC m=+34.111201603 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:36.460075 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.459856 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:55:52.459846946 +0000 UTC m=+34.111227379 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:36.661336 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:36.661297 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:36.661518 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.661490 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:36.661518 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.661516 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:36.661625 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.661529 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:36.661625 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.661597 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:55:52.661576665 +0000 UTC m=+34.312957102 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:36.899253 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:36.899214 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:36.899418 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:36.899267 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:36.899418 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.899362 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:36.899547 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:36.899472 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:37.899037 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:37.899005 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:37.899438 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:37.899141 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:38.900017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:38.899984 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:38.900465 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:38.900103 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:38.900465 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:38.900156 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:38.900465 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:38.900224 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:39.899393 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:39.899348 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:39.899577 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:39.899481 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:40.898763 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:40.898723 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:40.899322 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:40.898829 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:40.899322 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:40.898883 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:40.899322 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:40.898975 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:41.898350 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.898326 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:41.898489 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:41.898427 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:41.968421 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.968390 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="5d060bbc257c2c7887c46b974dfe77a98b94f04626bcf380a635fb8a2cc469a5" exitCode=0 Jul 17 18:55:41.969126 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.968471 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"5d060bbc257c2c7887c46b974dfe77a98b94f04626bcf380a635fb8a2cc469a5"} Jul 17 18:55:41.970071 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.970044 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" event={"ID":"04120778-d92c-40bd-8d4e-e300c09c44fe","Type":"ContainerStarted","Data":"8e2c9d0b04a3b1fa960953e77d6a29e3090f0077a2028ed5597c7688ca2f35f2"} Jul 17 18:55:41.972634 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.972614 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"b94612a57aaafa0ac7e430d38c8abb37a4067ff593f5f3ae16209ede3c7f3a20"} Jul 17 18:55:41.972720 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.972686 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"7a353c4d72f18f1544db7d8c825e308a28fab38581a2439828bbf165747b2b82"} Jul 17 18:55:41.972774 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.972732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"597bf79bd671ddf9aca3be276eab0d6b66f49f269fcacde4f340d92e9c489c41"} Jul 17 18:55:41.972774 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.972762 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"9d8b3e8b3ede1655c91f058d48883354a474b024052297f64c400aa3657cff8b"} Jul 17 18:55:41.972844 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.972795 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"c1623692d40f7e44fe72b4c28030bd3cdd6bd719e792b1193f5ef12160a033ec"} Jul 17 18:55:41.974040 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.974021 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" event={"ID":"0b57b1f4-92c2-4f9c-942c-d7a8375324dd","Type":"ContainerStarted","Data":"d9b83e8358e509d435280cf40978054bb90b9663241bf9b9b6418b7524df7421"} Jul 17 18:55:41.975814 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.975792 2584 generic.go:356] "Generic (PLEG): container finished" podID="5e1580e7866c089718b70d4daea63a50" containerID="50bf4d710770310af67e665f0bdc8db014bd5b5f694b76e329734be94ea52d3b" exitCode=0 Jul 17 18:55:41.975906 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.975856 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" event={"ID":"5e1580e7866c089718b70d4daea63a50","Type":"ContainerDied","Data":"50bf4d710770310af67e665f0bdc8db014bd5b5f694b76e329734be94ea52d3b"} Jul 17 18:55:41.977221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.977202 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-rpcgw" event={"ID":"aa36ca8f-467b-4d37-8eae-125febadac44","Type":"ContainerStarted","Data":"a1e8bedd3d9a35d7d0f86cc4f3fc7d5c2bbf51ae5379785e8390caca4a79c6a2"} Jul 17 18:55:41.978489 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.978466 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-h9bzt" event={"ID":"dbcb13f3-7841-445c-a433-5234b674aeca","Type":"ContainerStarted","Data":"8994486f00adbbac627ba6e5474ca7f689845cece03f29585718aaee214b4e70"} Jul 17 18:55:41.979781 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.979751 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-8q9k6" event={"ID":"d0a06bcd-e178-4db3-891d-ec3d90901d1e","Type":"ContainerStarted","Data":"841bbebce3df3794396825c0adb72acfaf6e65cd6b7ab7b3900b7c55d1b10643"} Jul 17 18:55:41.981259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.981237 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-jhjxm" event={"ID":"792eb6c8-4a63-4882-8985-10b8bcf3a1b9","Type":"ContainerStarted","Data":"167162afcebf90255da9f8ad6f0ed070ff4add1da226913fb9633bd5887342ea"} Jul 17 18:55:41.982747 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:41.982728 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" event={"ID":"50351eb0dda3c09fbc3ee35a82fae765","Type":"ContainerStarted","Data":"0e951ad70e65be43b4e352b18fbf0faf09e320389177956cd9727421c4b9b910"} Jul 17 18:55:42.013937 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.013884 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-8q9k6" podStartSLOduration=3.157516387 podStartE2EDuration="23.013869501s" totalImagesPullingTime="19.856353114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.013685497 +0000 UTC m=+23.665065941" watchObservedRunningTime="2026-07-17 18:55:42.013869501 +0000 UTC m=+23.665249956" Jul 17 18:55:42.047023 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.046980 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-rpcgw" podStartSLOduration=2.799112609 podStartE2EDuration="23.046966299s" totalImagesPullingTime="20.24785369s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.046627836 +0000 UTC m=+23.698008291" watchObservedRunningTime="2026-07-17 18:55:42.046966299 +0000 UTC m=+23.698346754" Jul 17 18:55:42.047189 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.047166 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-h9bzt" podStartSLOduration=2.99125142 podStartE2EDuration="23.047160822s" totalImagesPullingTime="20.055909402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.029808462 +0000 UTC m=+23.681188918" watchObservedRunningTime="2026-07-17 18:55:42.047160822 +0000 UTC m=+23.698541274" Jul 17 18:55:42.061203 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.061159 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-133-230.ec2.internal" podStartSLOduration=22.061146912 podStartE2EDuration="22.061146912s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.061056925 +0000 UTC m=+23.712437380" watchObservedRunningTime="2026-07-17 18:55:42.061146912 +0000 UTC m=+23.712527421" Jul 17 18:55:42.075879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.075843 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-jhjxm" podStartSLOduration=3.177191003 podStartE2EDuration="23.07583021s" totalImagesPullingTime="19.898639207s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.075460076 +0000 UTC m=+23.726840550" watchObservedRunningTime="2026-07-17 18:55:42.07583021 +0000 UTC m=+23.727210664" Jul 17 18:55:42.094423 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.094374 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-s5x6p" podStartSLOduration=2.988187584 podStartE2EDuration="23.094355871s" totalImagesPullingTime="20.106168287s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:42.094098058 +0000 UTC m=+23.745478514" watchObservedRunningTime="2026-07-17 18:55:42.094355871 +0000 UTC m=+23.745736327" Jul 17 18:55:42.899346 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.899092 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:42.899502 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.899153 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:42.899502 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:42.899402 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:42.899502 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:42.899474 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:42.988465 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.988429 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"e819fed761b51fb247272eb8e033cc73b167be5562b66698ca7034007f9e50d3"} Jul 17 18:55:42.990010 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:42.989979 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-fr466" event={"ID":"250faa54-3627-40ca-b09a-3587e23e29cc","Type":"ContainerStarted","Data":"f7c6bbff7bc23ecf4f3f1d4ffaee9374e0f7561fa6449a0fc22dc70d54dfa396"} Jul 17 18:55:43.092011 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.091992 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jhjxm_792eb6c8-4a63-4882-8985-10b8bcf3a1b9/node-ca/0.log" Jul 17 18:55:43.163697 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.163672 2584 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 17 18:55:43.781413 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.781320 2584 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-17T18:55:43.163694273Z","UUID":"26cea44a-1c3a-4459-8fac-669ff4074a6c","Handler":null,"Name":"","Endpoint":""} Jul 17 18:55:43.783073 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.783046 2584 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 17 18:55:43.783181 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.783084 2584 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 17 18:55:43.898671 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.898629 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:43.898859 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:43.898767 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:43.993248 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.993219 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" event={"ID":"5e1580e7866c089718b70d4daea63a50","Type":"ContainerStarted","Data":"943f27090bbee35dee23920c676f516d703adc0b1d8d215f31449e3743bbdc03"} Jul 17 18:55:43.994878 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:43.994836 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" event={"ID":"0b57b1f4-92c2-4f9c-942c-d7a8375324dd","Type":"ContainerStarted","Data":"e72ff0f5b416966a4ef5d4426552a7b99a5be66a186eb4cfd2e87139eeb4c9e9"} Jul 17 18:55:44.015513 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:44.013154 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-fr466" podStartSLOduration=4.89647532 podStartE2EDuration="25.013136517s" totalImagesPullingTime="20.116661197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:43.026931936 +0000 UTC m=+24.678312390" watchObservedRunningTime="2026-07-17 18:55:44.013136517 +0000 UTC m=+25.664516973" Jul 17 18:55:44.015513 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:44.013957 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-133-230.ec2.internal" podStartSLOduration=24.013940929 podStartE2EDuration="24.013940929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:44.013090281 +0000 UTC m=+25.664470737" watchObservedRunningTime="2026-07-17 18:55:44.013940929 +0000 UTC m=+25.665321395" Jul 17 18:55:44.898784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:44.898737 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:44.898965 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:44.898738 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:44.898965 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:44.898891 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:44.899240 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:44.899220 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:45.001071 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:45.001033 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"13fe372a722e68e4e6c966adb60c20cbc74f4e0ea2405eacde348bbb26c07331"} Jul 17 18:55:45.003226 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:45.003198 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" event={"ID":"0b57b1f4-92c2-4f9c-942c-d7a8375324dd","Type":"ContainerStarted","Data":"2b0b350c6ea8eb47638026bc41f688ff389c0ae8f1517707327a17c46a276d32"} Jul 17 18:55:45.899098 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:45.899060 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:45.899274 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:45.899172 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:46.070344 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:46.070306 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:46.071133 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:46.071112 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:46.089445 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:46.089390 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-x9tqn" podStartSLOduration=4.520070389 podStartE2EDuration="27.089373933s" totalImagesPullingTime="22.569303544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:45.024019237 +0000 UTC m=+26.675399716" watchObservedRunningTime="2026-07-17 18:55:46.089373933 +0000 UTC m=+27.740754387" Jul 17 18:55:46.902443 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:46.902373 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:46.902443 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:46.902390 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:46.902713 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:46.902492 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:46.902713 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:46.902660 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:47.007889 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:47.007861 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:47.008387 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:47.008368 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-8q9k6" Jul 17 18:55:47.899275 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:47.899043 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:47.899900 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:47.899289 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:48.011543 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.011513 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="70ca1cefc7c61b8cbb95eb9e083629e1f9f4b6728a56ecbf86885b76e39fdf23" exitCode=0 Jul 17 18:55:48.011716 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.011565 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"70ca1cefc7c61b8cbb95eb9e083629e1f9f4b6728a56ecbf86885b76e39fdf23"} Jul 17 18:55:48.015390 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.015361 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" event={"ID":"e52695ee-b0ac-4f2d-9c0e-4ee432280f05","Type":"ContainerStarted","Data":"ecc37ea7c8905059846041ce0882b26b820a928350ae43e9ebadd428dd693de2"} Jul 17 18:55:48.061728 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.061687 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" podStartSLOduration=8.906593336 podStartE2EDuration="29.061668158s" totalImagesPullingTime="20.155074822s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:48.061437319 +0000 UTC m=+29.712817773" watchObservedRunningTime="2026-07-17 18:55:48.061668158 +0000 UTC m=+29.713048613" Jul 17 18:55:48.899698 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.899664 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:48.899698 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:48.899705 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:48.900213 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:48.899780 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:48.900213 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:48.899889 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:49.017608 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.017586 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:49.017787 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.017658 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:49.017787 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.017696 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:49.033911 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.033889 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:49.034223 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.034204 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:55:49.481272 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.481018 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f67qc"] Jul 17 18:55:49.481425 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.481313 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:49.481425 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:49.481397 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:49.481509 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.481479 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lc7dh"] Jul 17 18:55:49.481581 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.481568 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:49.481713 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:49.481695 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:49.485432 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.485410 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6jwkm"] Jul 17 18:55:49.485556 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:49.485483 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:49.485556 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:49.485545 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:50.021128 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:50.021095 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="b9adf95c00d0ad9859503d74e1e5ffb9f4e6f7da2447f28b165184eee3316c12" exitCode=0 Jul 17 18:55:50.021487 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:50.021182 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"b9adf95c00d0ad9859503d74e1e5ffb9f4e6f7da2447f28b165184eee3316c12"} Jul 17 18:55:50.898744 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:50.898638 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:50.898878 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:50.898759 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:50.898878 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:50.898806 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:50.898968 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:50.898891 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:50.898968 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:50.898915 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:50.899050 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:50.899032 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:52.026829 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.026793 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="8fe9eb0f72b95887b431fcbc2bd0bcceae4651e3306735f7bbb87e2d7fe9da64" exitCode=0 Jul 17 18:55:52.027179 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.026843 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"8fe9eb0f72b95887b431fcbc2bd0bcceae4651e3306735f7bbb87e2d7fe9da64"} Jul 17 18:55:52.480001 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.479919 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:52.480001 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.479969 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:52.480187 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.480057 2584 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:52.480187 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.480095 2584 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:52.480187 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.480121 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret podName:c014f0b8-1ba7-40e4-861a-c27dc7c871b0 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:24.480106348 +0000 UTC m=+66.131486780 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret") pod "global-pull-secret-syncer-f67qc" (UID: "c014f0b8-1ba7-40e4-861a-c27dc7c871b0") : object "kube-system"/"original-pull-secret" not registered Jul 17 18:55:52.480187 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.480144 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs podName:1b173869-5333-421b-8a0f-f8dfc9a72faf nodeName:}" failed. No retries permitted until 2026-07-17 18:56:24.48012829 +0000 UTC m=+66.131508738 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs") pod "network-metrics-daemon-lc7dh" (UID: "1b173869-5333-421b-8a0f-f8dfc9a72faf") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 17 18:55:52.682203 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.682167 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:52.682356 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.682314 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 17 18:55:52.682356 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.682340 2584 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 17 18:55:52.682356 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.682354 2584 projected.go:196] Error preparing data for projected volume kube-api-access-k8bm4 for pod openshift-network-diagnostics/network-check-target-6jwkm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:52.682529 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.682409 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4 podName:30aa960f-cf39-4fcd-b805-1748b2731da0 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:24.682395874 +0000 UTC m=+66.333776308 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-k8bm4" (UniqueName: "kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4") pod "network-check-target-6jwkm" (UID: "30aa960f-cf39-4fcd-b805-1748b2731da0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 17 18:55:52.898942 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.898863 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:52.899104 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.898863 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:52.899104 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.898978 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:52.899104 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:52.898863 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:52.899104 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.899056 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:52.899243 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:52.899140 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:54.898507 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:54.898477 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:54.898507 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:54.898495 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:54.899117 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:54.898482 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:54.899117 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:54.898610 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:54.899222 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:54.899185 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:54.899357 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:54.899329 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:56.898952 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:56.898734 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:56.899320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:56.898734 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:56.899320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:56.898750 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:56.899320 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:56.899024 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:56.899320 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:56.899096 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:56.899320 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:56.899169 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:57.043388 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:57.043358 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="ede72acd589b23076af06b25bb23c44207802b1623203c9df9effefe9122a938" exitCode=0 Jul 17 18:55:57.043536 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:57.043407 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"ede72acd589b23076af06b25bb23c44207802b1623203c9df9effefe9122a938"} Jul 17 18:55:58.048811 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:58.048777 2584 generic.go:356] "Generic (PLEG): container finished" podID="f8f76668-6cf7-48a3-88dc-998738fa44c4" containerID="6d363ec1c9801b8485fc02c49ba353b02eb1e7534078d07150a268390415d1ca" exitCode=0 Jul 17 18:55:58.049179 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:58.048818 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerDied","Data":"6d363ec1c9801b8485fc02c49ba353b02eb1e7534078d07150a268390415d1ca"} Jul 17 18:55:58.900092 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:58.900052 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:55:58.900278 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:58.900099 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:55:58.900278 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:58.900185 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:55:58.900278 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:58.900198 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:55:58.900278 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:58.900264 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:55:58.900488 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:55:58.900313 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:55:59.054025 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:59.053998 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-vn44d" event={"ID":"f8f76668-6cf7-48a3-88dc-998738fa44c4","Type":"ContainerStarted","Data":"f07ed10aa1932f932fc1686d1a21900929a03a17d2ecddf66248d07200156595"} Jul 17 18:55:59.077941 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:55:59.077896 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-vn44d" podStartSLOduration=7.979633987 podStartE2EDuration="40.077881844s" totalImagesPullingTime="32.098247857s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:55:59.077455491 +0000 UTC m=+40.728835953" watchObservedRunningTime="2026-07-17 18:55:59.077881844 +0000 UTC m=+40.729262299" Jul 17 18:56:00.899176 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:00.899136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:00.899548 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:00.899186 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:00.899548 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:00.899137 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:00.899548 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:00.899271 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:00.899548 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:00.899331 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:00.899548 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:00.899405 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:02.898795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:02.898767 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:02.899174 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:02.898804 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:02.899174 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:02.898890 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:02.899174 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:02.898930 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:02.899174 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:02.898963 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:02.899174 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:02.899018 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:04.898353 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:04.898321 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:04.898960 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:04.898354 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:04.899088 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:04.898975 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:04.899088 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:04.898354 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:04.899088 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:04.899030 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:04.899224 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:04.899114 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:06.898772 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:06.898726 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:06.899135 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:06.898787 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:06.899135 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:06.898744 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:06.899135 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:06.898862 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:06.899135 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:06.898910 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:06.899135 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:06.898985 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:08.899178 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:08.899149 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:08.899555 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:08.899228 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:08.899555 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:08.899248 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:08.899555 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:08.899380 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:08.899555 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:08.899425 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:08.899555 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:08.899501 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:10.898570 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:10.898542 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:10.899041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:10.898539 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:10.899041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:10.898694 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lc7dh" podUID="1b173869-5333-421b-8a0f-f8dfc9a72faf" Jul 17 18:56:10.899041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:10.898550 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:10.899041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:10.898732 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-f67qc" podUID="c014f0b8-1ba7-40e4-861a-c27dc7c871b0" Jul 17 18:56:10.899041 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:10.898801 2584 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-6jwkm" podUID="30aa960f-cf39-4fcd-b805-1748b2731da0" Jul 17 18:56:11.189091 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.189021 2584 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-133-230.ec2.internal" event="NodeReady" Jul 17 18:56:11.189247 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.189139 2584 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 17 18:56:11.245812 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.245783 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2vvzl"] Jul 17 18:56:11.263490 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.263464 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-s47l6"] Jul 17 18:56:11.263689 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.263571 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.266747 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.266717 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 17 18:56:11.267285 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.267265 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 17 18:56:11.267391 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.267362 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-kfsnb\"" Jul 17 18:56:11.289465 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.289445 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vvzl"] Jul 17 18:56:11.289595 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.289480 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s47l6"] Jul 17 18:56:11.289595 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.289574 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.292378 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.292358 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 17 18:56:11.292571 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.292556 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-lp744\"" Jul 17 18:56:11.292621 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.292596 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 17 18:56:11.292681 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.292621 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 17 18:56:11.318135 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.318102 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78bf9e23-a025-4b34-8199-9a733270ac55-config-volume\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.341588 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.341560 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-tmkdl"] Jul 17 18:56:11.354531 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.354506 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.356002 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.355975 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmkdl"] Jul 17 18:56:11.357371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.357336 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 17 18:56:11.357492 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.357387 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 17 18:56:11.357492 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.357338 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 17 18:56:11.357492 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.357392 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-d9hlx\"" Jul 17 18:56:11.357674 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.357511 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 17 18:56:11.418725 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418702 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78bf9e23-a025-4b34-8199-9a733270ac55-config-volume\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.418852 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418744 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjz4b\" (UniqueName: \"kubernetes.io/projected/c887fff7-76a2-4122-94ee-18a2ba272cde-kube-api-access-rjz4b\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.418852 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418763 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78bf9e23-a025-4b34-8199-9a733270ac55-metrics-tls\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.418852 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418781 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7s6t6\" (UniqueName: \"kubernetes.io/projected/78bf9e23-a025-4b34-8199-9a733270ac55-kube-api-access-7s6t6\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.418951 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bf9e23-a025-4b34-8199-9a733270ac55-tmp-dir\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.418951 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.418910 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c887fff7-76a2-4122-94ee-18a2ba272cde-cert\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.419232 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.419216 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/78bf9e23-a025-4b34-8199-9a733270ac55-config-volume\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.519535 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519505 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjz4b\" (UniqueName: \"kubernetes.io/projected/c887fff7-76a2-4122-94ee-18a2ba272cde-kube-api-access-rjz4b\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.519535 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519540 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-crio-socket\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.519732 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519559 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78bf9e23-a025-4b34-8199-9a733270ac55-metrics-tls\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.519732 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519677 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-data-volume\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.519732 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519715 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7s6t6\" (UniqueName: \"kubernetes.io/projected/78bf9e23-a025-4b34-8199-9a733270ac55-kube-api-access-7s6t6\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.519865 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519742 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.519865 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519793 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bf9e23-a025-4b34-8199-9a733270ac55-tmp-dir\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.519865 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519831 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c887fff7-76a2-4122-94ee-18a2ba272cde-cert\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.519865 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519861 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.520056 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.519906 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdlsh\" (UniqueName: \"kubernetes.io/projected/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-api-access-jdlsh\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.520114 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.520094 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78bf9e23-a025-4b34-8199-9a733270ac55-tmp-dir\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.523676 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.523628 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/78bf9e23-a025-4b34-8199-9a733270ac55-metrics-tls\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.523790 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.523774 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/c887fff7-76a2-4122-94ee-18a2ba272cde-cert\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.532176 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.532147 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7s6t6\" (UniqueName: \"kubernetes.io/projected/78bf9e23-a025-4b34-8199-9a733270ac55-kube-api-access-7s6t6\") pod \"dns-default-2vvzl\" (UID: \"78bf9e23-a025-4b34-8199-9a733270ac55\") " pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.532454 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.532437 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjz4b\" (UniqueName: \"kubernetes.io/projected/c887fff7-76a2-4122-94ee-18a2ba272cde-kube-api-access-rjz4b\") pod \"ingress-canary-s47l6\" (UID: \"c887fff7-76a2-4122-94ee-18a2ba272cde\") " pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.573847 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.573807 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:11.597371 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.597346 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-s47l6" Jul 17 18:56:11.620484 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.620457 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.620611 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.620517 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdlsh\" (UniqueName: \"kubernetes.io/projected/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-api-access-jdlsh\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.620698 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.620683 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-crio-socket\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.620760 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.620718 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-data-volume\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.620760 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.620746 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.621029 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.621001 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-crio-socket\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.621104 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.621082 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.621241 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.621224 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-data-volume\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.622984 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.622967 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.632259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.632227 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdlsh\" (UniqueName: \"kubernetes.io/projected/73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d-kube-api-access-jdlsh\") pod \"insights-runtime-extractor-tmkdl\" (UID: \"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d\") " pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.663358 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.663330 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-tmkdl" Jul 17 18:56:11.797955 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.797924 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2vvzl"] Jul 17 18:56:11.801103 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:11.801077 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78bf9e23_a025_4b34_8199_9a733270ac55.slice/crio-3fb49a33f02371e8f395c1a81cc9c0fd1eca49a67e1e30a6a8eb9ce2a22cc454 WatchSource:0}: Error finding container 3fb49a33f02371e8f395c1a81cc9c0fd1eca49a67e1e30a6a8eb9ce2a22cc454: Status 404 returned error can't find the container with id 3fb49a33f02371e8f395c1a81cc9c0fd1eca49a67e1e30a6a8eb9ce2a22cc454 Jul 17 18:56:11.806437 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.806412 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-s47l6"] Jul 17 18:56:11.809176 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:11.809153 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc887fff7_76a2_4122_94ee_18a2ba272cde.slice/crio-eaa0f007d9c2fc9adffc740690d20c12523727a3cee44263d0778a02d0709a93 WatchSource:0}: Error finding container eaa0f007d9c2fc9adffc740690d20c12523727a3cee44263d0778a02d0709a93: Status 404 returned error can't find the container with id eaa0f007d9c2fc9adffc740690d20c12523727a3cee44263d0778a02d0709a93 Jul 17 18:56:11.849684 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:11.849628 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-tmkdl"] Jul 17 18:56:11.853638 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:11.853613 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73ac3ab2_6bb5_42e0_ab4a_b80949ac9a1d.slice/crio-e19a0200e7e85738ce89a9664d2529c6585860e4fd1663f12efe31a63ecd2060 WatchSource:0}: Error finding container e19a0200e7e85738ce89a9664d2529c6585860e4fd1663f12efe31a63ecd2060: Status 404 returned error can't find the container with id e19a0200e7e85738ce89a9664d2529c6585860e4fd1663f12efe31a63ecd2060 Jul 17 18:56:12.083329 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.083241 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmkdl" event={"ID":"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d","Type":"ContainerStarted","Data":"e19a0200e7e85738ce89a9664d2529c6585860e4fd1663f12efe31a63ecd2060"} Jul 17 18:56:12.084501 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.084470 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s47l6" event={"ID":"c887fff7-76a2-4122-94ee-18a2ba272cde","Type":"ContainerStarted","Data":"eaa0f007d9c2fc9adffc740690d20c12523727a3cee44263d0778a02d0709a93"} Jul 17 18:56:12.085463 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.085440 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvzl" event={"ID":"78bf9e23-a025-4b34-8199-9a733270ac55","Type":"ContainerStarted","Data":"3fb49a33f02371e8f395c1a81cc9c0fd1eca49a67e1e30a6a8eb9ce2a22cc454"} Jul 17 18:56:12.900886 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.900852 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:12.901977 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.901216 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:12.901977 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.901531 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:12.906576 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.906553 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 18:56:12.908255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.907377 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqhtg\"" Jul 17 18:56:12.908255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.907588 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n57jp\"" Jul 17 18:56:12.908255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.907762 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 18:56:12.908255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.907951 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 18:56:12.908255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:12.908111 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 18:56:14.355903 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.355867 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:14.357669 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.357625 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.361838 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.361817 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 18:56:14.362047 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.362029 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 18:56:14.362251 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.362237 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 18:56:14.362915 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.362886 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 18:56:14.363161 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.363144 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 18:56:14.363427 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.363404 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 18:56:14.363687 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.363672 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bdvh2\"" Jul 17 18:56:14.363967 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.363949 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 18:56:14.371814 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.371790 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438676 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438727 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz7vk\" (UniqueName: \"kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438759 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438828 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438859 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.438997 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.438887 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.539975 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.539933 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.540158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.539988 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.540158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.540019 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.540158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.540074 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.540158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.540097 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dz7vk\" (UniqueName: \"kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.540158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.540127 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.541895 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.540893 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.541895 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.541852 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.542088 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.541903 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.546121 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.546090 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.546243 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.546166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.551307 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.551281 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz7vk\" (UniqueName: \"kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk\") pod \"console-86bdcb557-f7hk7\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:14.674553 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:14.674462 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:16.545800 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.545763 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:56:16.565266 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.565233 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:56:16.565427 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.565345 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.573681 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.573660 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 18:56:16.656255 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656223 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656264 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656355 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656414 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w84xh\" (UniqueName: \"kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656564 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656443 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656564 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656483 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.656564 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.656508 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757227 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757411 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757274 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757411 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757321 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757539 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757500 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w84xh\" (UniqueName: \"kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757619 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757560 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757619 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.757741 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757622 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.758220 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.757997 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.758390 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.758365 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.758464 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.758392 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.758603 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.758584 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.760222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.760194 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.760840 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.760817 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.767455 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.767431 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w84xh\" (UniqueName: \"kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh\") pod \"console-c599489c8-44dl6\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:16.876517 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:16.876446 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:17.794138 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:17.790872 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:17.798555 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:17.798530 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde49ac0f_7b0f_413f_9fb5_9aa67c316d50.slice/crio-50807b73a4adca65c0d6237abf0899a994f9877c403433e03b13b16c3b5897c3 WatchSource:0}: Error finding container 50807b73a4adca65c0d6237abf0899a994f9877c403433e03b13b16c3b5897c3: Status 404 returned error can't find the container with id 50807b73a4adca65c0d6237abf0899a994f9877c403433e03b13b16c3b5897c3 Jul 17 18:56:17.818073 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:17.813553 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:56:17.822812 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:17.822770 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61ab76a6_14c5_40ae_8d47_1b26132d86da.slice/crio-6427f606b41228a077ffa0d51408a746ac4406715c909a836944b7f318b4edca WatchSource:0}: Error finding container 6427f606b41228a077ffa0d51408a746ac4406715c909a836944b7f318b4edca: Status 404 returned error can't find the container with id 6427f606b41228a077ffa0d51408a746ac4406715c909a836944b7f318b4edca Jul 17 18:56:18.104004 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.103967 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvzl" event={"ID":"78bf9e23-a025-4b34-8199-9a733270ac55","Type":"ContainerStarted","Data":"1441c679537c4c834a69a406335c08b4615abd92d0058206c3d14efbe64c7df6"} Jul 17 18:56:18.104186 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.104026 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2vvzl" event={"ID":"78bf9e23-a025-4b34-8199-9a733270ac55","Type":"ContainerStarted","Data":"3b41e753a2a07bfde67eb3de5d57ca1591acadc4a1161e40e944784c868c9c38"} Jul 17 18:56:18.104186 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.104088 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:18.105208 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.105178 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdcb557-f7hk7" event={"ID":"de49ac0f-7b0f-413f-9fb5-9aa67c316d50","Type":"ContainerStarted","Data":"50807b73a4adca65c0d6237abf0899a994f9877c403433e03b13b16c3b5897c3"} Jul 17 18:56:18.106566 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.106546 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmkdl" event={"ID":"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d","Type":"ContainerStarted","Data":"60f2aefb063155d15b7b9c95ff6e6fd29e3476596b8e4d6838f953590679b8b4"} Jul 17 18:56:18.107867 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.107844 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-s47l6" event={"ID":"c887fff7-76a2-4122-94ee-18a2ba272cde","Type":"ContainerStarted","Data":"e56c8a9ade167d219e1601d2a8d8965443776604f2af3ee96a3b21f524064620"} Jul 17 18:56:18.109218 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.109198 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c599489c8-44dl6" event={"ID":"61ab76a6-14c5-40ae-8d47-1b26132d86da","Type":"ContainerStarted","Data":"6427f606b41228a077ffa0d51408a746ac4406715c909a836944b7f318b4edca"} Jul 17 18:56:18.125769 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.125726 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2vvzl" podStartSLOduration=1.292072385 podStartE2EDuration="7.125712404s" totalImagesPullingTime="5.833640019s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:18.124237001 +0000 UTC m=+59.775617457" watchObservedRunningTime="2026-07-17 18:56:18.125712404 +0000 UTC m=+59.777092837" Jul 17 18:56:18.147343 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:18.147295 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-s47l6" podStartSLOduration=1.32009377 podStartE2EDuration="7.14727925s" totalImagesPullingTime="5.82718548s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:18.146084633 +0000 UTC m=+59.797465089" watchObservedRunningTime="2026-07-17 18:56:18.14727925 +0000 UTC m=+59.798659705" Jul 17 18:56:21.035793 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:21.035759 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hhckh" Jul 17 18:56:23.127521 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.127126 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdcb557-f7hk7" event={"ID":"de49ac0f-7b0f-413f-9fb5-9aa67c316d50","Type":"ContainerStarted","Data":"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee"} Jul 17 18:56:23.129334 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.129305 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmkdl" event={"ID":"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d","Type":"ContainerStarted","Data":"11b6c2a5c744495bc5d3c172619923972b0ae8a28d391a685ec4c7c66235a5f2"} Jul 17 18:56:23.131055 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.131027 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c599489c8-44dl6" event={"ID":"61ab76a6-14c5-40ae-8d47-1b26132d86da","Type":"ContainerStarted","Data":"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662"} Jul 17 18:56:23.146738 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.146680 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86bdcb557-f7hk7" podStartSLOduration=4.290424117 podStartE2EDuration="9.146660272s" totalImagesPullingTime="4.856236155s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:23.146143151 +0000 UTC m=+64.797523625" watchObservedRunningTime="2026-07-17 18:56:23.146660272 +0000 UTC m=+64.798040724" Jul 17 18:56:23.164889 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.164827 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-c599489c8-44dl6" podStartSLOduration=2.328004012 podStartE2EDuration="7.164814435s" totalImagesPullingTime="4.836810423s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:23.163854342 +0000 UTC m=+64.815234822" watchObservedRunningTime="2026-07-17 18:56:23.164814435 +0000 UTC m=+64.816194887" Jul 17 18:56:23.380566 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.380479 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz"] Jul 17 18:56:23.382660 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.382620 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.386070 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.386043 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 17 18:56:23.389968 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.389939 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 17 18:56:23.390134 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.390114 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-fbmcs\"" Jul 17 18:56:23.390210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.390055 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 17 18:56:23.390210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.390118 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 17 18:56:23.390326 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.389968 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 17 18:56:23.396224 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.396201 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz"] Jul 17 18:56:23.417716 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.417677 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lkkq\" (UniqueName: \"kubernetes.io/projected/48281b7d-0e8f-42bc-9cef-149f27a842f8-kube-api-access-9lkkq\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.417852 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.417737 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.417852 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.417838 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48281b7d-0e8f-42bc-9cef-149f27a842f8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.417961 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.417889 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.519511 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.519467 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lkkq\" (UniqueName: \"kubernetes.io/projected/48281b7d-0e8f-42bc-9cef-149f27a842f8-kube-api-access-9lkkq\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.519710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.519583 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.519710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.519625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48281b7d-0e8f-42bc-9cef-149f27a842f8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.519833 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:23.519719 2584 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 17 18:56:23.519833 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:23.519795 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls podName:48281b7d-0e8f-42bc-9cef-149f27a842f8 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:24.019775509 +0000 UTC m=+65.671155941 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls") pod "prometheus-operator-5945b5cfd-v6mcz" (UID: "48281b7d-0e8f-42bc-9cef-149f27a842f8") : secret "prometheus-operator-tls" not found Jul 17 18:56:23.519944 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.519725 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.520392 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.520367 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/48281b7d-0e8f-42bc-9cef-149f27a842f8-metrics-client-ca\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.522354 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.522335 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:23.531970 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:23.531945 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lkkq\" (UniqueName: \"kubernetes.io/projected/48281b7d-0e8f-42bc-9cef-149f27a842f8-kube-api-access-9lkkq\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:24.024038 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.024005 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:24.026758 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.026734 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/48281b7d-0e8f-42bc-9cef-149f27a842f8-prometheus-operator-tls\") pod \"prometheus-operator-5945b5cfd-v6mcz\" (UID: \"48281b7d-0e8f-42bc-9cef-149f27a842f8\") " pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:24.297133 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.297051 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" Jul 17 18:56:24.434309 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.434279 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz"] Jul 17 18:56:24.528049 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.528015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:24.528207 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.528070 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:24.531084 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.531060 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 17 18:56:24.531084 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.531060 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 17 18:56:24.541271 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.541250 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/1b173869-5333-421b-8a0f-f8dfc9a72faf-metrics-certs\") pod \"network-metrics-daemon-lc7dh\" (UID: \"1b173869-5333-421b-8a0f-f8dfc9a72faf\") " pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:24.541389 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.541250 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/c014f0b8-1ba7-40e4-861a-c27dc7c871b0-original-pull-secret\") pod \"global-pull-secret-syncer-f67qc\" (UID: \"c014f0b8-1ba7-40e4-861a-c27dc7c871b0\") " pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:24.623107 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.623031 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-f67qc" Jul 17 18:56:24.633969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.633949 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-n57jp\"" Jul 17 18:56:24.641320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.641295 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lc7dh" Jul 17 18:56:24.675638 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.675610 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:24.675768 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.675687 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:24.680972 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.680951 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:24.730354 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.730325 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:24.733680 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.733636 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 17 18:56:24.744076 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.744052 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 17 18:56:24.754342 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.754314 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k8bm4\" (UniqueName: \"kubernetes.io/projected/30aa960f-cf39-4fcd-b805-1748b2731da0-kube-api-access-k8bm4\") pod \"network-check-target-6jwkm\" (UID: \"30aa960f-cf39-4fcd-b805-1748b2731da0\") " pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:24.918836 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.918758 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-bqhtg\"" Jul 17 18:56:24.927044 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:24.927019 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:25.143871 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:25.143845 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:25.996550 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:25.996506 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48281b7d_0e8f_42bc_9cef_149f27a842f8.slice/crio-b20635a181e8da0b47b2d6d26f91f117d732230965685f3d5beb89030f7faf62 WatchSource:0}: Error finding container b20635a181e8da0b47b2d6d26f91f117d732230965685f3d5beb89030f7faf62: Status 404 returned error can't find the container with id b20635a181e8da0b47b2d6d26f91f117d732230965685f3d5beb89030f7faf62 Jul 17 18:56:26.144605 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.144564 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-tmkdl" event={"ID":"73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d","Type":"ContainerStarted","Data":"c4754cdf9c3073bfceb14755752375ca64f9fc1bec762d0e0a39ee3a76b0713d"} Jul 17 18:56:26.146572 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.146505 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" event={"ID":"48281b7d-0e8f-42bc-9cef-149f27a842f8","Type":"ContainerStarted","Data":"b20635a181e8da0b47b2d6d26f91f117d732230965685f3d5beb89030f7faf62"} Jul 17 18:56:26.157265 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.157241 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lc7dh"] Jul 17 18:56:26.162344 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:26.162317 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b173869_5333_421b_8a0f_f8dfc9a72faf.slice/crio-d663c2ee3a407dfca42dbd5977867a8806f3a83e00042e8e0c86ad3507c59a0a WatchSource:0}: Error finding container d663c2ee3a407dfca42dbd5977867a8806f3a83e00042e8e0c86ad3507c59a0a: Status 404 returned error can't find the container with id d663c2ee3a407dfca42dbd5977867a8806f3a83e00042e8e0c86ad3507c59a0a Jul 17 18:56:26.171073 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.171034 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-tmkdl" podStartSLOduration=1.140359568 podStartE2EDuration="15.171018299s" totalImagesPullingTime="14.030658731s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:56:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:26.170064532 +0000 UTC m=+67.821445022" watchObservedRunningTime="2026-07-17 18:56:26.171018299 +0000 UTC m=+67.822398754" Jul 17 18:56:26.378999 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.378910 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-f67qc"] Jul 17 18:56:26.379819 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.379699 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-6jwkm"] Jul 17 18:56:26.382698 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:26.382668 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30aa960f_cf39_4fcd_b805_1748b2731da0.slice/crio-a95d92d22c56f4c1c6040d43341123094ab49913a16f2d568d92b7c4e840e9e2 WatchSource:0}: Error finding container a95d92d22c56f4c1c6040d43341123094ab49913a16f2d568d92b7c4e840e9e2: Status 404 returned error can't find the container with id a95d92d22c56f4c1c6040d43341123094ab49913a16f2d568d92b7c4e840e9e2 Jul 17 18:56:26.383426 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:26.383327 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc014f0b8_1ba7_40e4_861a_c27dc7c871b0.slice/crio-af94a85a0d0856f35efb2ba51b7af984a54df0369b83f2d1338e4aa242e94974 WatchSource:0}: Error finding container af94a85a0d0856f35efb2ba51b7af984a54df0369b83f2d1338e4aa242e94974: Status 404 returned error can't find the container with id af94a85a0d0856f35efb2ba51b7af984a54df0369b83f2d1338e4aa242e94974 Jul 17 18:56:26.876897 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.876866 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:26.877082 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.876955 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:26.883208 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:26.883178 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:27.153323 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:27.153205 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f67qc" event={"ID":"c014f0b8-1ba7-40e4-861a-c27dc7c871b0","Type":"ContainerStarted","Data":"af94a85a0d0856f35efb2ba51b7af984a54df0369b83f2d1338e4aa242e94974"} Jul 17 18:56:27.154843 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:27.154814 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6jwkm" event={"ID":"30aa960f-cf39-4fcd-b805-1748b2731da0","Type":"ContainerStarted","Data":"a95d92d22c56f4c1c6040d43341123094ab49913a16f2d568d92b7c4e840e9e2"} Jul 17 18:56:27.156959 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:27.156934 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc7dh" event={"ID":"1b173869-5333-421b-8a0f-f8dfc9a72faf","Type":"ContainerStarted","Data":"d663c2ee3a407dfca42dbd5977867a8806f3a83e00042e8e0c86ad3507c59a0a"} Jul 17 18:56:27.163182 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:27.162953 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:56:27.225619 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:27.225584 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:28.114992 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.114896 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2vvzl" Jul 17 18:56:28.166670 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.166593 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc7dh" event={"ID":"1b173869-5333-421b-8a0f-f8dfc9a72faf","Type":"ContainerStarted","Data":"b8de1e39eccc2b1c8fc5245dcd5760bcf8074d8077d7bd90457fd0011cca3c6c"} Jul 17 18:56:28.167073 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.166678 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lc7dh" event={"ID":"1b173869-5333-421b-8a0f-f8dfc9a72faf","Type":"ContainerStarted","Data":"e05542db2727afb5ecc16d45c9a00f689bdb5c990835e5b1b30313ac67488bfd"} Jul 17 18:56:28.171191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.171113 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" event={"ID":"48281b7d-0e8f-42bc-9cef-149f27a842f8","Type":"ContainerStarted","Data":"8ba51ec65ec70f75819113115c32a23f248cddfbc7ef100b279457b8c348c406"} Jul 17 18:56:28.171191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.171168 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" event={"ID":"48281b7d-0e8f-42bc-9cef-149f27a842f8","Type":"ContainerStarted","Data":"27c4554cf67cab12afd82166c0161aa0eef10f1914d0921e26fc17b05e4d4caf"} Jul 17 18:56:28.185159 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.183910 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lc7dh" podStartSLOduration=67.425436794 podStartE2EDuration="1m9.183888845s" totalImagesPullingTime="1.758452051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:28.182756682 +0000 UTC m=+69.834137137" watchObservedRunningTime="2026-07-17 18:56:28.183888845 +0000 UTC m=+69.835269303" Jul 17 18:56:28.206238 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:28.206175 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-5945b5cfd-v6mcz" podStartSLOduration=3.284402671 podStartE2EDuration="5.206150901s" totalImagesPullingTime="1.92174823s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:28.204552302 +0000 UTC m=+69.855932758" watchObservedRunningTime="2026-07-17 18:56:28.206150901 +0000 UTC m=+69.857531357" Jul 17 18:56:29.719535 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.719499 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc"] Jul 17 18:56:29.722183 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.722159 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.725794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.725683 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 17 18:56:29.725794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.725691 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-hr6nr\"" Jul 17 18:56:29.725992 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.725844 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 17 18:56:29.733269 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.733243 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-s57j5"] Jul 17 18:56:29.736441 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.736417 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc"] Jul 17 18:56:29.736567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.736516 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-b7h29"] Jul 17 18:56:29.736612 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.736592 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.738709 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.738688 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.739557 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.739495 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 17 18:56:29.739557 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.739536 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 17 18:56:29.739740 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.739711 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 17 18:56:29.740200 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.740045 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-sj6hh\"" Jul 17 18:56:29.741320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.741300 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 17 18:56:29.741420 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.741355 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 17 18:56:29.741513 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.741493 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 17 18:56:29.741612 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.741594 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-fk854\"" Jul 17 18:56:29.750717 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.750691 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-b7h29"] Jul 17 18:56:29.776527 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.776494 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bd52955-7d4b-4ef5-b42b-7808e526fa63-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.776700 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.776551 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.776700 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.776634 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pph9f\" (UniqueName: \"kubernetes.io/projected/7bd52955-7d4b-4ef5-b42b-7808e526fa63-kube-api-access-pph9f\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.776802 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.776710 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.877397 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877350 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-sys\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877538 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877401 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877538 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877462 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-metrics-client-ca\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877538 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877497 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877538 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877522 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877546 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/08ec5972-0c66-407f-aff1-5e238be5106e-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877620 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-wtmp\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877683 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877734 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddkjc\" (UniqueName: \"kubernetes.io/projected/08ec5972-0c66-407f-aff1-5e238be5106e-kube-api-access-ddkjc\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877784 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877777 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bd52955-7d4b-4ef5-b42b-7808e526fa63-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877810 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-accelerators-collector-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877842 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877868 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-textfile\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877895 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ksw4w\" (UniqueName: \"kubernetes.io/projected/1fbb2101-cc97-46e4-a085-5ca56a6a017f-kube-api-access-ksw4w\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.877993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.877981 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pph9f\" (UniqueName: \"kubernetes.io/projected/7bd52955-7d4b-4ef5-b42b-7808e526fa63-kube-api-access-pph9f\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.878257 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.878011 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.878257 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.878053 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-root\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.878257 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.878163 2584 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 17 18:56:29.878257 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.878231 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls podName:7bd52955-7d4b-4ef5-b42b-7808e526fa63 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:30.378211317 +0000 UTC m=+72.029591754 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-5zbdc" (UID: "7bd52955-7d4b-4ef5-b42b-7808e526fa63") : secret "openshift-state-metrics-tls" not found Jul 17 18:56:29.878517 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.878494 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7bd52955-7d4b-4ef5-b42b-7808e526fa63-metrics-client-ca\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.880465 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.880440 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.887489 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.887463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pph9f\" (UniqueName: \"kubernetes.io/projected/7bd52955-7d4b-4ef5-b42b-7808e526fa63-kube-api-access-pph9f\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:29.979560 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979475 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-sys\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979560 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979560 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979543 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-sys\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979560 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979554 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-metrics-client-ca\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979597 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979620 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979669 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979709 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/08ec5972-0c66-407f-aff1-5e238be5106e-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-wtmp\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.979759 2584 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.979785 2584 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.979828 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls podName:08ec5972-0c66-407f-aff1-5e238be5106e nodeName:}" failed. No retries permitted until 2026-07-17 18:56:30.479803653 +0000 UTC m=+72.131184090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls") pod "kube-state-metrics-56c5594674-b7h29" (UID: "08ec5972-0c66-407f-aff1-5e238be5106e") : secret "kube-state-metrics-tls" not found Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:29.979864 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls podName:1fbb2101-cc97-46e4-a085-5ca56a6a017f nodeName:}" failed. No retries permitted until 2026-07-17 18:56:30.479843624 +0000 UTC m=+72.131224062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls") pod "node-exporter-s57j5" (UID: "1fbb2101-cc97-46e4-a085-5ca56a6a017f") : secret "node-exporter-tls" not found Jul 17 18:56:29.979920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979761 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979924 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddkjc\" (UniqueName: \"kubernetes.io/projected/08ec5972-0c66-407f-aff1-5e238be5106e-kube-api-access-ddkjc\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.979974 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-accelerators-collector-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980030 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-textfile\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980053 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980102 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ksw4w\" (UniqueName: \"kubernetes.io/projected/1fbb2101-cc97-46e4-a085-5ca56a6a017f-kube-api-access-ksw4w\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980172 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-wtmp\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980190 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-root\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980291 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-metrics-client-ca\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/1fbb2101-cc97-46e4-a085-5ca56a6a017f-root\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980419 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980318 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/08ec5972-0c66-407f-aff1-5e238be5106e-volume-directive-shadow\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.980857 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980519 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.980857 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980537 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-textfile\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980857 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980789 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-accelerators-collector-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.980986 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.980965 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/08ec5972-0c66-407f-aff1-5e238be5106e-metrics-client-ca\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.982477 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.982452 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:29.983022 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.983003 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.989754 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.989725 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddkjc\" (UniqueName: \"kubernetes.io/projected/08ec5972-0c66-407f-aff1-5e238be5106e-kube-api-access-ddkjc\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:29.990228 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:29.990203 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ksw4w\" (UniqueName: \"kubernetes.io/projected/1fbb2101-cc97-46e4-a085-5ca56a6a017f-kube-api-access-ksw4w\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:30.384356 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.384271 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:30.384510 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:30.384443 2584 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 17 18:56:30.384563 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:30.384521 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls podName:7bd52955-7d4b-4ef5-b42b-7808e526fa63 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:31.384500705 +0000 UTC m=+73.035881151 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls") pod "openshift-state-metrics-79f6958cdf-5zbdc" (UID: "7bd52955-7d4b-4ef5-b42b-7808e526fa63") : secret "openshift-state-metrics-tls" not found Jul 17 18:56:30.484936 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.484896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:30.484936 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.484937 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:30.487914 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.487883 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/1fbb2101-cc97-46e4-a085-5ca56a6a017f-node-exporter-tls\") pod \"node-exporter-s57j5\" (UID: \"1fbb2101-cc97-46e4-a085-5ca56a6a017f\") " pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:30.488093 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.488071 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/08ec5972-0c66-407f-aff1-5e238be5106e-kube-state-metrics-tls\") pod \"kube-state-metrics-56c5594674-b7h29\" (UID: \"08ec5972-0c66-407f-aff1-5e238be5106e\") " pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:30.649948 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.649870 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-s57j5" Jul 17 18:56:30.657660 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.657620 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" Jul 17 18:56:30.806304 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.806269 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:56:30.812801 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.812773 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.817518 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.817486 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 18:56:30.817665 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.817541 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 18:56:30.817665 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.817492 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 18:56:30.817665 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.817491 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 18:56:30.818213 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.818078 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 18:56:30.818213 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.818187 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 18:56:30.818514 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.818495 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nk9rk\"" Jul 17 18:56:30.818913 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.818897 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 18:56:30.819943 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.819925 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 18:56:30.824273 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.824249 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 18:56:30.845923 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.845893 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:56:30.888903 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.888842 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cb2j\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.888903 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.888899 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889064 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.888970 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889064 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889006 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889064 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889032 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889074 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889093 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889116 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889149 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889173 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889187 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889372 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889241 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.889372 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.889260 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990158 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990123 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990171 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990198 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990264 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990288 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990330 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990317 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cb2j\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990589 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990351 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990698 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:30.990673 2584 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 17 18:56:30.990778 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:30.990764 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls podName:d58aff2c-7396-4024-9e1b-2274dd8134c0 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:31.490744395 +0000 UTC m=+73.142124831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0") : secret "alertmanager-main-tls" not found Jul 17 18:56:30.990962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.990963 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.990962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.991008 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.991294 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.991066 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.991294 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.991090 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.991294 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.991126 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.991294 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.991210 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.993567 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.993542 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.993803 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.993775 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994081 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.993870 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994081 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:30.993926 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle podName:d58aff2c-7396-4024-9e1b-2274dd8134c0 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:31.493907645 +0000 UTC m=+73.145288088 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0") : configmap references non-existent config key: ca-bundle.crt Jul 17 18:56:30.994081 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.993990 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994285 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.994135 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.994306 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994817 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.994760 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.994817 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.994763 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:30.995229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:30.995206 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:31.000308 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.000286 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cb2j\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:31.347619 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:31.347581 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fbb2101_cc97_46e4_a085_5ca56a6a017f.slice/crio-6c63723bec10c26849bb1d8c892041bc596fe876ac943c49b0974d809b498ffd WatchSource:0}: Error finding container 6c63723bec10c26849bb1d8c892041bc596fe876ac943c49b0974d809b498ffd: Status 404 returned error can't find the container with id 6c63723bec10c26849bb1d8c892041bc596fe876ac943c49b0974d809b498ffd Jul 17 18:56:31.393940 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.393905 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:31.400611 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.400571 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/7bd52955-7d4b-4ef5-b42b-7808e526fa63-openshift-state-metrics-tls\") pod \"openshift-state-metrics-79f6958cdf-5zbdc\" (UID: \"7bd52955-7d4b-4ef5-b42b-7808e526fa63\") " pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:31.490469 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.490390 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-56c5594674-b7h29"] Jul 17 18:56:31.492401 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:31.492377 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08ec5972_0c66_407f_aff1_5e238be5106e.slice/crio-6abb7fcc6b551bbde42806fa2b08fd3faf8e536a5ba7032e012f59dcf1a88c52 WatchSource:0}: Error finding container 6abb7fcc6b551bbde42806fa2b08fd3faf8e536a5ba7032e012f59dcf1a88c52: Status 404 returned error can't find the container with id 6abb7fcc6b551bbde42806fa2b08fd3faf8e536a5ba7032e012f59dcf1a88c52 Jul 17 18:56:31.494569 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.494543 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:31.494683 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.494612 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:31.494770 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:31.494753 2584 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 17 18:56:31.494828 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:31.494814 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls podName:d58aff2c-7396-4024-9e1b-2274dd8134c0 nodeName:}" failed. No retries permitted until 2026-07-17 18:56:32.494795324 +0000 UTC m=+74.146175764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0") : secret "alertmanager-main-tls" not found Jul 17 18:56:31.495546 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.495518 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:31.532537 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.532483 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" Jul 17 18:56:31.666085 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:31.666054 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc"] Jul 17 18:56:31.669324 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:31.669299 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7bd52955_7d4b_4ef5_b42b_7808e526fa63.slice/crio-baf3220a9707b9ee95120049cca5ec91c98e25eab58121114c993d74e4b675e9 WatchSource:0}: Error finding container baf3220a9707b9ee95120049cca5ec91c98e25eab58121114c993d74e4b675e9: Status 404 returned error can't find the container with id baf3220a9707b9ee95120049cca5ec91c98e25eab58121114c993d74e4b675e9 Jul 17 18:56:32.189049 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.189003 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" event={"ID":"08ec5972-0c66-407f-aff1-5e238be5106e","Type":"ContainerStarted","Data":"6abb7fcc6b551bbde42806fa2b08fd3faf8e536a5ba7032e012f59dcf1a88c52"} Jul 17 18:56:32.190305 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.190269 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s57j5" event={"ID":"1fbb2101-cc97-46e4-a085-5ca56a6a017f","Type":"ContainerStarted","Data":"6c63723bec10c26849bb1d8c892041bc596fe876ac943c49b0974d809b498ffd"} Jul 17 18:56:32.191787 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.191755 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-f67qc" event={"ID":"c014f0b8-1ba7-40e4-861a-c27dc7c871b0","Type":"ContainerStarted","Data":"ff268d5760886731e6086f664718081a303a570a2fb1dd8d7b7d18f45c44128d"} Jul 17 18:56:32.193441 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.193419 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-6jwkm" event={"ID":"30aa960f-cf39-4fcd-b805-1748b2731da0","Type":"ContainerStarted","Data":"5d284559693b6838dac11e3e86bde9621736185bc17aeeb5687407b4a9f938a1"} Jul 17 18:56:32.193550 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.193471 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:56:32.195687 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.195663 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" event={"ID":"7bd52955-7d4b-4ef5-b42b-7808e526fa63","Type":"ContainerStarted","Data":"e04354e073d9a342667920fb2d3a1011f43b42d0f8379cf1d577528503e1d99f"} Jul 17 18:56:32.195766 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.195719 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" event={"ID":"7bd52955-7d4b-4ef5-b42b-7808e526fa63","Type":"ContainerStarted","Data":"3f10f7cffa2cf84e3cc7ddc0f3dca182a937baaa38df9f961ccb92c77fa998fe"} Jul 17 18:56:32.195766 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.195739 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" event={"ID":"7bd52955-7d4b-4ef5-b42b-7808e526fa63","Type":"ContainerStarted","Data":"baf3220a9707b9ee95120049cca5ec91c98e25eab58121114c993d74e4b675e9"} Jul 17 18:56:32.236058 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.236006 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-6jwkm" podStartSLOduration=68.272441269 podStartE2EDuration="1m13.235987733s" totalImagesPullingTime="4.963546464s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:32.235391838 +0000 UTC m=+73.886772295" watchObservedRunningTime="2026-07-17 18:56:32.235987733 +0000 UTC m=+73.887368188" Jul 17 18:56:32.236637 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.236608 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-f67qc" podStartSLOduration=68.265796599 podStartE2EDuration="1m13.236600166s" totalImagesPullingTime="4.970803567s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 18:55:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:32.210409613 +0000 UTC m=+73.861790088" watchObservedRunningTime="2026-07-17 18:56:32.236600166 +0000 UTC m=+73.887980620" Jul 17 18:56:32.507749 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.507666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:32.510370 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.510329 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:32.624814 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.624751 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:56:32.819032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.818835 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:56:32.824005 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.823973 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk"] Jul 17 18:56:32.827682 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.827663 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.830741 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.830712 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 17 18:56:32.830879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.830757 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-jpzx8\"" Jul 17 18:56:32.830879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.830779 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 17 18:56:32.831063 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.831044 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 17 18:56:32.831124 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.831103 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-31hesbd0euagg\"" Jul 17 18:56:32.831192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.831045 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 17 18:56:32.831243 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.831191 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 17 18:56:32.841328 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.841304 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk"] Jul 17 18:56:32.913979 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.913945 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.913979 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.913991 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013525df-e413-4025-8ffb-d39f2f8b1daa-metrics-client-ca\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914013 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-grpc-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914061 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914095 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914152 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jrx4q\" (UniqueName: \"kubernetes.io/projected/013525df-e413-4025-8ffb-d39f2f8b1daa-kube-api-access-jrx4q\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:32.914222 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:32.914211 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015461 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015429 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015461 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015465 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015666 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jrx4q\" (UniqueName: \"kubernetes.io/projected/013525df-e413-4025-8ffb-d39f2f8b1daa-kube-api-access-jrx4q\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015765 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015739 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015831 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015812 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015882 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015860 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013525df-e413-4025-8ffb-d39f2f8b1daa-metrics-client-ca\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015946 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015889 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-grpc-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.015946 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.015928 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.016706 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.016612 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/013525df-e413-4025-8ffb-d39f2f8b1daa-metrics-client-ca\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.018717 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.018691 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.018813 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.018769 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.018813 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.018783 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.019259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.019238 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.019380 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.019362 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.019479 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.019463 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/013525df-e413-4025-8ffb-d39f2f8b1daa-secret-grpc-tls\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.022372 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:33.022349 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd58aff2c_7396_4024_9e1b_2274dd8134c0.slice/crio-b8fe2b1577eb8bec213515927d940308fbfd8fce45c9b1f8162a3df093c32d72 WatchSource:0}: Error finding container b8fe2b1577eb8bec213515927d940308fbfd8fce45c9b1f8162a3df093c32d72: Status 404 returned error can't find the container with id b8fe2b1577eb8bec213515927d940308fbfd8fce45c9b1f8162a3df093c32d72 Jul 17 18:56:33.029907 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.029883 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jrx4q\" (UniqueName: \"kubernetes.io/projected/013525df-e413-4025-8ffb-d39f2f8b1daa-kube-api-access-jrx4q\") pod \"thanos-querier-6d69ff8d68-tl4qk\" (UID: \"013525df-e413-4025-8ffb-d39f2f8b1daa\") " pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.138684 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.138582 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:33.200974 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.200938 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"b8fe2b1577eb8bec213515927d940308fbfd8fce45c9b1f8162a3df093c32d72"} Jul 17 18:56:33.202876 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.202848 2584 generic.go:356] "Generic (PLEG): container finished" podID="1fbb2101-cc97-46e4-a085-5ca56a6a017f" containerID="fe05d048d2b8d4559f1df839449f4ed3b388220e69a02021b7cb152a16663cc4" exitCode=0 Jul 17 18:56:33.203023 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.202949 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s57j5" event={"ID":"1fbb2101-cc97-46e4-a085-5ca56a6a017f","Type":"ContainerDied","Data":"fe05d048d2b8d4559f1df839449f4ed3b388220e69a02021b7cb152a16663cc4"} Jul 17 18:56:33.560625 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:33.560590 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk"] Jul 17 18:56:33.564406 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:33.564372 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod013525df_e413_4025_8ffb_d39f2f8b1daa.slice/crio-b7dbb4ff67d6da5cb486eb1002717e7d603f2171df59cb79ff4274b9983ad68d WatchSource:0}: Error finding container b7dbb4ff67d6da5cb486eb1002717e7d603f2171df59cb79ff4274b9983ad68d: Status 404 returned error can't find the container with id b7dbb4ff67d6da5cb486eb1002717e7d603f2171df59cb79ff4274b9983ad68d Jul 17 18:56:34.024781 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.024746 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-55f8d9db6b-xkln2"] Jul 17 18:56:34.030194 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.029759 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.032688 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.032666 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-4p4t5\"" Jul 17 18:56:34.032794 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.032666 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-ad2vj79rtq8rc\"" Jul 17 18:56:34.033023 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.033006 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 17 18:56:34.033346 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.033332 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 17 18:56:34.033550 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.033538 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 17 18:56:34.033729 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.033710 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 17 18:56:34.039053 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.039026 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55f8d9db6b-xkln2"] Jul 17 18:56:34.128617 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128576 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-client-certs\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.128617 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-metrics-server-audit-profiles\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.128879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128684 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.128879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128749 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-tls\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.128879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128802 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b264b71e-1111-4220-8ba6-d1c75cbc2d55-audit-log\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.128879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssth8\" (UniqueName: \"kubernetes.io/projected/b264b71e-1111-4220-8ba6-d1c75cbc2d55-kube-api-access-ssth8\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.129086 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.128979 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-client-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.211157 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.211110 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s57j5" event={"ID":"1fbb2101-cc97-46e4-a085-5ca56a6a017f","Type":"ContainerStarted","Data":"3501515504ca41ab988164d25edb51de1543af05d275750a234a2b0c50863598"} Jul 17 18:56:34.211600 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.211192 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-s57j5" event={"ID":"1fbb2101-cc97-46e4-a085-5ca56a6a017f","Type":"ContainerStarted","Data":"da5d1ac5ea8b49d5640b31af5f330b5724be2aca8afa95988a2cd28b993b5ddd"} Jul 17 18:56:34.212395 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.212365 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"b7dbb4ff67d6da5cb486eb1002717e7d603f2171df59cb79ff4274b9983ad68d"} Jul 17 18:56:34.214708 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.214673 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" event={"ID":"7bd52955-7d4b-4ef5-b42b-7808e526fa63","Type":"ContainerStarted","Data":"7285c60983fefb5bcaf679e19cf655876a664c40934eca1cafa4581e250299c0"} Jul 17 18:56:34.217103 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.217075 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" event={"ID":"08ec5972-0c66-407f-aff1-5e238be5106e","Type":"ContainerStarted","Data":"8dba28a80d0e5c810152368fb0191ea74e4c8de2b0708daa33a4934395b95830"} Jul 17 18:56:34.217180 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.217125 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" event={"ID":"08ec5972-0c66-407f-aff1-5e238be5106e","Type":"ContainerStarted","Data":"904bcef0730ec47f8c30a9047e492374c92b0d9639056e8f5227bc6e2749c946"} Jul 17 18:56:34.217180 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.217145 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" event={"ID":"08ec5972-0c66-407f-aff1-5e238be5106e","Type":"ContainerStarted","Data":"5fcc5bb143983911e999c41d4e93b691dbb021b4d3b9875dc7f9865cad781a51"} Jul 17 18:56:34.229462 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229440 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b264b71e-1111-4220-8ba6-d1c75cbc2d55-audit-log\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229583 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229485 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssth8\" (UniqueName: \"kubernetes.io/projected/b264b71e-1111-4220-8ba6-d1c75cbc2d55-kube-api-access-ssth8\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229583 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229546 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-client-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229705 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229590 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-client-certs\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229705 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229619 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-metrics-server-audit-profiles\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229705 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229675 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229880 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229705 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-tls\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.229989 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.229968 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/b264b71e-1111-4220-8ba6-d1c75cbc2d55-audit-log\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.230593 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.230567 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-metrics-server-audit-profiles\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.230939 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.230912 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b264b71e-1111-4220-8ba6-d1c75cbc2d55-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.232719 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.232695 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-client-certs\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.233233 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.233181 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-s57j5" podStartSLOduration=4.241731704 podStartE2EDuration="5.233164167s" totalImagesPullingTime="918.171604ms" totalInitContainerRuntime="73.260859ms" isStatefulPod=true podCreationTimestamp="2026-07-17 18:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:34.232945069 +0000 UTC m=+75.884325525" watchObservedRunningTime="2026-07-17 18:56:34.233164167 +0000 UTC m=+75.884544623" Jul 17 18:56:34.233355 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.233320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-client-ca-bundle\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.233818 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.233793 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/b264b71e-1111-4220-8ba6-d1c75cbc2d55-secret-metrics-server-tls\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.239229 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.239208 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssth8\" (UniqueName: \"kubernetes.io/projected/b264b71e-1111-4220-8ba6-d1c75cbc2d55-kube-api-access-ssth8\") pod \"metrics-server-55f8d9db6b-xkln2\" (UID: \"b264b71e-1111-4220-8ba6-d1c75cbc2d55\") " pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.253077 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.253030 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-79f6958cdf-5zbdc" podStartSLOduration=3.641608971 podStartE2EDuration="5.253015547s" totalImagesPullingTime="1.611406576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:34.252572293 +0000 UTC m=+75.903952749" watchObservedRunningTime="2026-07-17 18:56:34.253015547 +0000 UTC m=+75.904396029" Jul 17 18:56:34.279948 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.279854 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-56c5594674-b7h29" podStartSLOduration=3.382019368 podStartE2EDuration="5.279831745s" totalImagesPullingTime="1.897812377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:34.279046346 +0000 UTC m=+75.930426803" watchObservedRunningTime="2026-07-17 18:56:34.279831745 +0000 UTC m=+75.931212200" Jul 17 18:56:34.342575 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.342547 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:34.484520 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.484495 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-55f8d9db6b-xkln2"] Jul 17 18:56:34.487085 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:34.487060 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb264b71e_1111_4220_8ba6_d1c75cbc2d55.slice/crio-1d5c88236204dc64617c0119f3846a1f7fd3af2266200baa8ad40a4657e873d3 WatchSource:0}: Error finding container 1d5c88236204dc64617c0119f3846a1f7fd3af2266200baa8ad40a4657e873d3: Status 404 returned error can't find the container with id 1d5c88236204dc64617c0119f3846a1f7fd3af2266200baa8ad40a4657e873d3 Jul 17 18:56:34.570685 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.570577 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv"] Jul 17 18:56:34.574164 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.574136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.577300 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.577281 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 17 18:56:34.577704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.577681 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mbtmc\"" Jul 17 18:56:34.584324 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.584304 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv"] Jul 17 18:56:34.734546 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.734511 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.734732 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.734587 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2psc\" (UniqueName: \"kubernetes.io/projected/b7664783-2ba8-4c0a-826e-c13bbf19b76c-kube-api-access-h2psc\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.835901 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.835805 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.835901 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.835879 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2psc\" (UniqueName: \"kubernetes.io/projected/b7664783-2ba8-4c0a-826e-c13bbf19b76c-kube-api-access-h2psc\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.836128 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:34.835975 2584 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 17 18:56:34.836128 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:34.836055 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert podName:b7664783-2ba8-4c0a-826e-c13bbf19b76c nodeName:}" failed. No retries permitted until 2026-07-17 18:56:35.336031921 +0000 UTC m=+76.987412374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert") pod "monitoring-plugin-7b7b7bc8bf-cjfzv" (UID: "b7664783-2ba8-4c0a-826e-c13bbf19b76c") : secret "monitoring-plugin-cert" not found Jul 17 18:56:34.845994 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.845962 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2psc\" (UniqueName: \"kubernetes.io/projected/b7664783-2ba8-4c0a-826e-c13bbf19b76c-kube-api-access-h2psc\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:34.923389 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.923355 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-84fbf498b5-qlnlt"] Jul 17 18:56:34.926876 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.926856 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:34.930105 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930082 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 17 18:56:34.930105 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930092 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 17 18:56:34.930282 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930125 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 17 18:56:34.930343 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930282 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 17 18:56:34.930506 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930479 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 17 18:56:34.930506 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.930496 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fqfrg\"" Jul 17 18:56:34.935785 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.934984 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 17 18:56:34.942576 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:34.942553 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84fbf498b5-qlnlt"] Jul 17 18:56:35.037721 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037683 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.037915 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-federate-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.037915 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037812 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-metrics-client-ca\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.037915 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037842 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.038082 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037944 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.038082 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.037994 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.038082 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.038058 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-serving-certs-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.038239 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.038092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvtsr\" (UniqueName: \"kubernetes.io/projected/03728292-ebe0-46ae-991d-048ef0cd68a7-kube-api-access-cvtsr\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139152 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139053 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139152 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139118 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139172 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-serving-certs-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139204 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvtsr\" (UniqueName: \"kubernetes.io/projected/03728292-ebe0-46ae-991d-048ef0cd68a7-kube-api-access-cvtsr\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139244 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-federate-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139313 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-metrics-client-ca\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.139363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.139335 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.140200 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.140149 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-serving-certs-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.140420 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.140395 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-trusted-ca-bundle\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.140604 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.140582 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/03728292-ebe0-46ae-991d-048ef0cd68a7-metrics-client-ca\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.142359 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.142333 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-telemeter-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.142776 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.142754 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.142901 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.142879 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-federate-client-tls\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.142963 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.142898 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/03728292-ebe0-46ae-991d-048ef0cd68a7-secret-telemeter-client\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.151384 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.151350 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvtsr\" (UniqueName: \"kubernetes.io/projected/03728292-ebe0-46ae-991d-048ef0cd68a7-kube-api-access-cvtsr\") pod \"telemeter-client-84fbf498b5-qlnlt\" (UID: \"03728292-ebe0-46ae-991d-048ef0cd68a7\") " pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.223585 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.223537 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" event={"ID":"b264b71e-1111-4220-8ba6-d1c75cbc2d55","Type":"ContainerStarted","Data":"1d5c88236204dc64617c0119f3846a1f7fd3af2266200baa8ad40a4657e873d3"} Jul 17 18:56:35.225469 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.225439 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8" exitCode=0 Jul 17 18:56:35.225615 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.225588 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8"} Jul 17 18:56:35.240150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.240122 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" Jul 17 18:56:35.344279 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.341467 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:35.350536 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.350486 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7664783-2ba8-4c0a-826e-c13bbf19b76c-monitoring-plugin-cert\") pod \"monitoring-plugin-7b7b7bc8bf-cjfzv\" (UID: \"b7664783-2ba8-4c0a-826e-c13bbf19b76c\") " pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:35.404307 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.403879 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-84fbf498b5-qlnlt"] Jul 17 18:56:35.487570 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.487541 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:35.778584 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.778545 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:56:35.782167 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.782136 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.798159 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.798087 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:56:35.947860 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947714 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.947860 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947774 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zxt6w\" (UniqueName: \"kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.947860 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947794 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.947860 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947831 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.948150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947913 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.948150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947955 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:35.948150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:35.947982 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.032364 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.032278 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:56:36.036729 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.036706 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.041312 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.040940 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 17 18:56:36.041312 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041249 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 17 18:56:36.041542 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041504 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 17 18:56:36.041681 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041622 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 17 18:56:36.041778 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041751 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 17 18:56:36.041987 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041956 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 17 18:56:36.042045 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.042016 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 17 18:56:36.042129 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.041967 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 17 18:56:36.042767 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.042745 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 17 18:56:36.043117 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043013 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 17 18:56:36.043117 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043063 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-buro0hbftskms\"" Jul 17 18:56:36.043202 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043146 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 17 18:56:36.043390 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043368 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ct8sz\"" Jul 17 18:56:36.043503 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043472 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 17 18:56:36.043944 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.043926 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 17 18:56:36.044900 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.044882 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 17 18:56:36.048709 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048676 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.048820 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zxt6w\" (UniqueName: \"kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.048820 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048767 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.048925 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048826 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.048925 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.048925 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048894 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.049063 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.048926 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.050171 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.050152 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.050296 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.050160 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.050396 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.050225 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.050789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.050745 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.052321 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.052283 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.053016 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.052991 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.061698 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.061674 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:56:36.063586 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.063559 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zxt6w\" (UniqueName: \"kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w\") pod \"console-64dc679f6d-phfzg\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.097834 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.097554 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:36.150336 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150298 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150519 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b6xrf\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150519 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150401 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150519 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150506 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150711 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150583 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150711 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150618 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150711 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150697 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150883 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150724 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.150975 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150951 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151052 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.150992 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151052 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151047 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151156 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151074 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151156 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151098 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151156 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151136 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151340 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151161 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151340 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151221 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151340 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151247 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151340 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151285 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151340 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151312 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.151559 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.151339 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.232389 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.232347 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" event={"ID":"03728292-ebe0-46ae-991d-048ef0cd68a7","Type":"ContainerStarted","Data":"322ba173a9bbd36e51226ba00c4040d669b0fa776f29ac358a5ed6da064c24a3"} Jul 17 18:56:36.252850 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252808 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252890 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252921 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b6xrf\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.252989 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253012 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253064 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253143 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253198 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253220 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253247 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253270 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253293 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253317 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253847 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253338 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253847 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253372 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253847 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253400 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.253847 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.253529 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.254214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.254149 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.255949 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.255043 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.255949 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.255615 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.257275 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.256577 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.257599 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.257572 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.259018 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.259481 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.259597 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.259981 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260032 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.260013 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260361 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.260125 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260770 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.260540 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.260982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.260942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.261258 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.261217 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.261893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.261871 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.262739 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.262698 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.263043 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.263018 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.263886 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.263864 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.270937 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.270903 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b6xrf\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf\") pod \"prometheus-k8s-0\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.352087 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.352011 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:36.617314 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.617225 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv"] Jul 17 18:56:36.680533 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.679586 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:56:36.685517 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:36.685474 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod27c408de_184a_406d_bf6b_f13db23c37d2.slice/crio-c0a0e79f687c625bfa21b0281d6d1b9f17995f9353556bebb11f22748ca751ed WatchSource:0}: Error finding container c0a0e79f687c625bfa21b0281d6d1b9f17995f9353556bebb11f22748ca751ed: Status 404 returned error can't find the container with id c0a0e79f687c625bfa21b0281d6d1b9f17995f9353556bebb11f22748ca751ed Jul 17 18:56:36.709582 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:36.709504 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:56:36.720832 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:56:36.720805 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1273390c_3f28_4ac3_952f_6c8b1e068ead.slice/crio-a2d1c92cc00553268b616e08eea00c236b2b5a17a2790656fb6f1ec4c54d8933 WatchSource:0}: Error finding container a2d1c92cc00553268b616e08eea00c236b2b5a17a2790656fb6f1ec4c54d8933: Status 404 returned error can't find the container with id a2d1c92cc00553268b616e08eea00c236b2b5a17a2790656fb6f1ec4c54d8933 Jul 17 18:56:37.241239 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.241037 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"d41d6305b3a2cd0019153f6570082820a142ab31ab150d209bd3e697eda25fa0"} Jul 17 18:56:37.241239 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.241131 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"445fc570b132fa4891f08ce6a4d266552c235dc7e982c8f31256880d23e97e74"} Jul 17 18:56:37.241239 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.241174 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"2248aefc967832f18b165a320cffe583de653f5801a9dcbdff21b49b7c639315"} Jul 17 18:56:37.243068 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.243036 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" event={"ID":"b7664783-2ba8-4c0a-826e-c13bbf19b76c","Type":"ContainerStarted","Data":"5f52928a74a043de499139834def10dce48086a1824d9a1309f55b5d02898c7b"} Jul 17 18:56:37.245083 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.245056 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" event={"ID":"b264b71e-1111-4220-8ba6-d1c75cbc2d55","Type":"ContainerStarted","Data":"26b8f673b48491216a5ccf14d0e613ff32e579f73f0f70616b58339dce7c2efb"} Jul 17 18:56:37.247856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.247553 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="17a9418ee0fd08cfc56a9dc7982a5422cdba1981a6d4c74237033c3b97de6960" exitCode=0 Jul 17 18:56:37.247856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.247709 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"17a9418ee0fd08cfc56a9dc7982a5422cdba1981a6d4c74237033c3b97de6960"} Jul 17 18:56:37.247856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.247778 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"a2d1c92cc00553268b616e08eea00c236b2b5a17a2790656fb6f1ec4c54d8933"} Jul 17 18:56:37.251747 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.251720 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc679f6d-phfzg" event={"ID":"27c408de-184a-406d-bf6b-f13db23c37d2","Type":"ContainerStarted","Data":"90cd135d469e9bf5cc5baa6d24c52d16a057270e9567b5fb45b1513626d9667d"} Jul 17 18:56:37.251835 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.251771 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc679f6d-phfzg" event={"ID":"27c408de-184a-406d-bf6b-f13db23c37d2","Type":"ContainerStarted","Data":"c0a0e79f687c625bfa21b0281d6d1b9f17995f9353556bebb11f22748ca751ed"} Jul 17 18:56:37.272039 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.271983 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" podStartSLOduration=1.345191116 podStartE2EDuration="3.271962834s" totalImagesPullingTime="1.926771718s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:37.270362731 +0000 UTC m=+78.921743187" watchObservedRunningTime="2026-07-17 18:56:37.271962834 +0000 UTC m=+78.923343290" Jul 17 18:56:37.332462 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:37.331314 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-64dc679f6d-phfzg" podStartSLOduration=2.331293434 podStartE2EDuration="2.331293434s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:37.330499694 +0000 UTC m=+78.981880150" watchObservedRunningTime="2026-07-17 18:56:37.331293434 +0000 UTC m=+78.982673890" Jul 17 18:56:39.267523 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.267468 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208"} Jul 17 18:56:39.267977 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.267554 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425"} Jul 17 18:56:39.267977 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.267580 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e"} Jul 17 18:56:39.271480 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.271455 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"368208cb378cf5f7f89bc95200cf221421d2f980ff76d27296f960499af5cad5"} Jul 17 18:56:39.271571 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.271511 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"8719311277f367d3673adb9663724f3d00ff6e13ee7dd31cc6cc9922929a536f"} Jul 17 18:56:39.271571 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.271536 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" event={"ID":"013525df-e413-4025-8ffb-d39f2f8b1daa","Type":"ContainerStarted","Data":"d8c67adabfd7ec04c0285921f0fafcad70724205372cc2be225fe3c6b3358c3d"} Jul 17 18:56:39.272363 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.272301 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:39.274444 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.274159 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" event={"ID":"b7664783-2ba8-4c0a-826e-c13bbf19b76c","Type":"ContainerStarted","Data":"381433f3f8368bc1e1d24c8d7aef1d0d0bf3ece62d4e01392d979e8b0117e2ef"} Jul 17 18:56:39.274444 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.274256 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:39.278603 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.278056 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" event={"ID":"03728292-ebe0-46ae-991d-048ef0cd68a7","Type":"ContainerStarted","Data":"c744ebf4d322a3e37ef4d5329bd194a8301c6909ea0ec89557c6c142cd9a4ca4"} Jul 17 18:56:39.278603 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.278107 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" event={"ID":"03728292-ebe0-46ae-991d-048ef0cd68a7","Type":"ContainerStarted","Data":"11518089b3fb7e1dd3c879ce6f889413760ccdf49d20c63ea48e9f9ecd2c0ba4"} Jul 17 18:56:39.278603 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.278135 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" event={"ID":"03728292-ebe0-46ae-991d-048ef0cd68a7","Type":"ContainerStarted","Data":"33d967ff63acde84acc73613a07a36ae0642bed4c7b4a8f533d0dbe0c1322dc5"} Jul 17 18:56:39.281218 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.281177 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" Jul 17 18:56:39.300893 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.300542 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" podStartSLOduration=2.367317628 podStartE2EDuration="7.300525485s" totalImagesPullingTime="4.933207857s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:32 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:39.299488582 +0000 UTC m=+80.950869039" watchObservedRunningTime="2026-07-17 18:56:39.300525485 +0000 UTC m=+80.951905939" Jul 17 18:56:39.325745 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.325218 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-84fbf498b5-qlnlt" podStartSLOduration=2.297171649 podStartE2EDuration="5.325196049s" totalImagesPullingTime="3.0280244s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:39.324370915 +0000 UTC m=+80.975751393" watchObservedRunningTime="2026-07-17 18:56:39.325196049 +0000 UTC m=+80.976576506" Jul 17 18:56:39.343407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:39.343329 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-7b7b7bc8bf-cjfzv" podStartSLOduration=3.086442233 podStartE2EDuration="5.343308334s" totalImagesPullingTime="2.256866101s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:39.342272765 +0000 UTC m=+80.993653270" watchObservedRunningTime="2026-07-17 18:56:39.343308334 +0000 UTC m=+80.994688787" Jul 17 18:56:40.118580 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:40.118542 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:56:40.288747 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:40.288708 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c"} Jul 17 18:56:40.289195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:40.288787 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71"} Jul 17 18:56:40.289195 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:40.288825 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerStarted","Data":"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4"} Jul 17 18:56:40.322332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:40.322248 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=5.376242261 podStartE2EDuration="10.3222319s" totalImagesPullingTime="4.928591207s" totalInitContainerRuntime="17.398432ms" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:40.321186629 +0000 UTC m=+81.972567086" watchObservedRunningTime="2026-07-17 18:56:40.3222319 +0000 UTC m=+81.973612355" Jul 17 18:56:41.299207 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:41.299182 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d69ff8d68-tl4qk" Jul 17 18:56:42.301025 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.300993 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"1393306fbac778da18ff37fa939268d476760296a786cf1fec30d3ef72305445"} Jul 17 18:56:42.301400 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.301060 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"97c1022b7ca98ecd5958b26997c7525184abf145ae0fcaa3661a3ad081d3de0e"} Jul 17 18:56:42.301400 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.301086 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"634ac45890b6d698ba57d965b7daf57bbcdaa4324f2ded8a5aa6e4cf6065d92e"} Jul 17 18:56:42.301400 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.301111 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"6e93eefab70e7f1fd4864a2d249d55420763b21e665333b5c1bc01fd0b86fc8d"} Jul 17 18:56:42.301400 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.301135 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"974549d1e7ccabe69504d4874d8e3e7d88ef9797e6ffbc3ee4893d4ea17ea589"} Jul 17 18:56:42.301400 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.301158 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerStarted","Data":"223b9ad56493e890203681a7fb9842b1cbd0305bfa9a4b719c79213e322cca2f"} Jul 17 18:56:42.334846 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:42.334797 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.323382427 podStartE2EDuration="7.334781906s" totalImagesPullingTime="3.979417361s" totalInitContainerRuntime="31.982118ms" isStatefulPod=false podCreationTimestamp="2026-07-17 18:56:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:56:42.33225298 +0000 UTC m=+83.983633465" watchObservedRunningTime="2026-07-17 18:56:42.334781906 +0000 UTC m=+83.986162361" Jul 17 18:56:46.098312 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:46.098279 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:56:46.352544 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:46.352455 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:56:50.808381 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:50.808350 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:56:53.205687 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:53.205609 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86bdcb557-f7hk7" podUID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" containerName="console" containerID="cri-o://fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee" gracePeriod=15 Jul 17 18:56:53.960948 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:53.960926 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:54.018938 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.018909 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.018938 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.018949 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.019214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.018981 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.019214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019035 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.019214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019073 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.019214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019104 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz7vk\" (UniqueName: \"kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk\") pod \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\" (UID: \"de49ac0f-7b0f-413f-9fb5-9aa67c316d50\") " Jul 17 18:56:54.019511 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019483 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:56:54.019590 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019481 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config" (OuterVolumeSpecName: "console-config") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:56:54.019590 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.019473 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca" (OuterVolumeSpecName: "service-ca") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:56:54.021543 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.021519 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:56:54.021729 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.021552 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk" (OuterVolumeSpecName: "kube-api-access-dz7vk") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "kube-api-access-dz7vk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:56:54.021729 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.021552 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "de49ac0f-7b0f-413f-9fb5-9aa67c316d50" (UID: "de49ac0f-7b0f-413f-9fb5-9aa67c316d50"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120362 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-oauth-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120388 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120397 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-service-ca\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120405 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz7vk\" (UniqueName: \"kubernetes.io/projected/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-kube-api-access-dz7vk\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120413 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-oauth-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.120430 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.120420 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/de49ac0f-7b0f-413f-9fb5-9aa67c316d50-console-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:56:54.343385 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.343340 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:54.343856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.343400 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:56:54.354670 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.354627 2584 generic.go:356] "Generic (PLEG): container finished" podID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" containerID="fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee" exitCode=0 Jul 17 18:56:54.354789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.354672 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdcb557-f7hk7" event={"ID":"de49ac0f-7b0f-413f-9fb5-9aa67c316d50","Type":"ContainerDied","Data":"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee"} Jul 17 18:56:54.354789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.354689 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86bdcb557-f7hk7" Jul 17 18:56:54.354789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.354721 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86bdcb557-f7hk7" event={"ID":"de49ac0f-7b0f-413f-9fb5-9aa67c316d50","Type":"ContainerDied","Data":"50807b73a4adca65c0d6237abf0899a994f9877c403433e03b13b16c3b5897c3"} Jul 17 18:56:54.354789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.354757 2584 scope.go:117] "RemoveContainer" containerID="fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee" Jul 17 18:56:54.363714 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.363696 2584 scope.go:117] "RemoveContainer" containerID="fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee" Jul 17 18:56:54.363998 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:56:54.363977 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee\": container with ID starting with fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee not found: ID does not exist" containerID="fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee" Jul 17 18:56:54.364073 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.364010 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee"} err="failed to get container status \"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee\": rpc error: code = NotFound desc = could not find container \"fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee\": container with ID starting with fbeda699a0704e9fd341c8eb0db88bc326e20e70eb96a03e05fa50fe7703e1ee not found: ID does not exist" Jul 17 18:56:54.381210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.381147 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:54.386924 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.386903 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86bdcb557-f7hk7"] Jul 17 18:56:54.903583 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:56:54.903550 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" path="/var/lib/kubelet/pods/de49ac0f-7b0f-413f-9fb5-9aa67c316d50/volumes" Jul 17 18:57:03.205579 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:03.205544 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-6jwkm" Jul 17 18:57:05.141891 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.141820 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-64dc679f6d-phfzg" podUID="27c408de-184a-406d-bf6b-f13db23c37d2" containerName="console" containerID="cri-o://90cd135d469e9bf5cc5baa6d24c52d16a057270e9567b5fb45b1513626d9667d" gracePeriod=15 Jul 17 18:57:05.404668 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.404565 2584 generic.go:356] "Generic (PLEG): container finished" podID="27c408de-184a-406d-bf6b-f13db23c37d2" containerID="90cd135d469e9bf5cc5baa6d24c52d16a057270e9567b5fb45b1513626d9667d" exitCode=0 Jul 17 18:57:05.404810 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.404661 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc679f6d-phfzg" event={"ID":"27c408de-184a-406d-bf6b-f13db23c37d2","Type":"ContainerDied","Data":"90cd135d469e9bf5cc5baa6d24c52d16a057270e9567b5fb45b1513626d9667d"} Jul 17 18:57:05.444422 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.444396 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:57:05.511920 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.511890 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512111 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.511933 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512111 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.511961 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512111 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512046 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512111 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512098 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512307 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512155 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zxt6w\" (UniqueName: \"kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512307 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512180 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle\") pod \"27c408de-184a-406d-bf6b-f13db23c37d2\" (UID: \"27c408de-184a-406d-bf6b-f13db23c37d2\") " Jul 17 18:57:05.512503 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512455 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca" (OuterVolumeSpecName: "service-ca") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:05.512503 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512452 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:05.512503 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512481 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config" (OuterVolumeSpecName: "console-config") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:05.512867 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.512812 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:05.514398 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.514361 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:05.514489 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.514395 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:05.514489 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.514459 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w" (OuterVolumeSpecName: "kube-api-access-zxt6w") pod "27c408de-184a-406d-bf6b-f13db23c37d2" (UID: "27c408de-184a-406d-bf6b-f13db23c37d2"). InnerVolumeSpecName "kube-api-access-zxt6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:05.613545 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613506 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zxt6w\" (UniqueName: \"kubernetes.io/projected/27c408de-184a-406d-bf6b-f13db23c37d2-kube-api-access-zxt6w\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613545 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613549 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-trusted-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613755 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613565 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613755 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613581 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/27c408de-184a-406d-bf6b-f13db23c37d2-console-oauth-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613755 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613596 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-service-ca\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613755 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613611 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-oauth-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:05.613755 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:05.613624 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/27c408de-184a-406d-bf6b-f13db23c37d2-console-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:06.410024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.409993 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64dc679f6d-phfzg" Jul 17 18:57:06.410433 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.409993 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64dc679f6d-phfzg" event={"ID":"27c408de-184a-406d-bf6b-f13db23c37d2","Type":"ContainerDied","Data":"c0a0e79f687c625bfa21b0281d6d1b9f17995f9353556bebb11f22748ca751ed"} Jul 17 18:57:06.410433 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.410149 2584 scope.go:117] "RemoveContainer" containerID="90cd135d469e9bf5cc5baa6d24c52d16a057270e9567b5fb45b1513626d9667d" Jul 17 18:57:06.433329 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.433298 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:57:06.435212 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.435189 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64dc679f6d-phfzg"] Jul 17 18:57:06.904448 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:06.904415 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="27c408de-184a-406d-bf6b-f13db23c37d2" path="/var/lib/kubelet/pods/27c408de-184a-406d-bf6b-f13db23c37d2/volumes" Jul 17 18:57:14.348994 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:14.348958 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:57:14.352837 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:14.352783 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-55f8d9db6b-xkln2" Jul 17 18:57:15.829327 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:15.829267 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-c599489c8-44dl6" podUID="61ab76a6-14c5-40ae-8d47-1b26132d86da" containerName="console" containerID="cri-o://2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662" gracePeriod=15 Jul 17 18:57:16.069029 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.069008 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:57:16.207496 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207467 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207505 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207534 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207559 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w84xh\" (UniqueName: \"kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207604 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207693 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207626 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.207962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.207701 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca\") pod \"61ab76a6-14c5-40ae-8d47-1b26132d86da\" (UID: \"61ab76a6-14c5-40ae-8d47-1b26132d86da\") " Jul 17 18:57:16.208095 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.208067 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config" (OuterVolumeSpecName: "console-config") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:16.208179 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.208081 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:16.208238 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.208187 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:16.208460 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.208239 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca" (OuterVolumeSpecName: "service-ca") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:16.210002 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.209973 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:16.210095 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.210008 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:16.210095 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.210006 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh" (OuterVolumeSpecName: "kube-api-access-w84xh") pod "61ab76a6-14c5-40ae-8d47-1b26132d86da" (UID: "61ab76a6-14c5-40ae-8d47-1b26132d86da"). InnerVolumeSpecName "kube-api-access-w84xh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:16.309079 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309045 2584 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309079 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309074 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w84xh\" (UniqueName: \"kubernetes.io/projected/61ab76a6-14c5-40ae-8d47-1b26132d86da-kube-api-access-w84xh\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309088 2584 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-trusted-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309101 2584 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-oauth-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309112 2584 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/61ab76a6-14c5-40ae-8d47-1b26132d86da-service-ca\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309124 2584 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-serving-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.309263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.309134 2584 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/61ab76a6-14c5-40ae-8d47-1b26132d86da-console-oauth-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:16.452130 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.452095 2584 generic.go:356] "Generic (PLEG): container finished" podID="61ab76a6-14c5-40ae-8d47-1b26132d86da" containerID="2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662" exitCode=0 Jul 17 18:57:16.452277 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.452142 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-c599489c8-44dl6" Jul 17 18:57:16.452277 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.452181 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c599489c8-44dl6" event={"ID":"61ab76a6-14c5-40ae-8d47-1b26132d86da","Type":"ContainerDied","Data":"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662"} Jul 17 18:57:16.452277 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.452237 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-c599489c8-44dl6" event={"ID":"61ab76a6-14c5-40ae-8d47-1b26132d86da","Type":"ContainerDied","Data":"6427f606b41228a077ffa0d51408a746ac4406715c909a836944b7f318b4edca"} Jul 17 18:57:16.452277 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.452259 2584 scope.go:117] "RemoveContainer" containerID="2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662" Jul 17 18:57:16.461407 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.461394 2584 scope.go:117] "RemoveContainer" containerID="2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662" Jul 17 18:57:16.461673 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:16.461631 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662\": container with ID starting with 2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662 not found: ID does not exist" containerID="2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662" Jul 17 18:57:16.461736 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.461682 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662"} err="failed to get container status \"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662\": rpc error: code = NotFound desc = could not find container \"2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662\": container with ID starting with 2083af010fdf00e7f0f61cc6e916ce10f4f20e58daac2935738c50d17dd39662 not found: ID does not exist" Jul 17 18:57:16.475287 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.475259 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:57:16.479092 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.479074 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-c599489c8-44dl6"] Jul 17 18:57:16.903889 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:16.903813 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="61ab76a6-14c5-40ae-8d47-1b26132d86da" path="/var/lib/kubelet/pods/61ab76a6-14c5-40ae-8d47-1b26132d86da/volumes" Jul 17 18:57:36.352331 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:36.352293 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:36.371861 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:36.371836 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:36.545603 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:36.545577 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:50.176149 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176112 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176400 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="alertmanager" containerID="cri-o://b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" gracePeriod=120 Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176475 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-metric" containerID="cri-o://66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" gracePeriod=120 Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176543 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="prom-label-proxy" containerID="cri-o://9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" gracePeriod=120 Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176557 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy" containerID="cri-o://6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" gracePeriod=120 Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176496 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-web" containerID="cri-o://6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" gracePeriod=120 Jul 17 18:57:50.176699 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.176529 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="config-reloader" containerID="cri-o://75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" gracePeriod=120 Jul 17 18:57:50.585348 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585316 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" exitCode=0 Jul 17 18:57:50.585348 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585342 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" exitCode=0 Jul 17 18:57:50.585348 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585350 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" exitCode=0 Jul 17 18:57:50.585348 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585357 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" exitCode=0 Jul 17 18:57:50.585607 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585395 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71"} Jul 17 18:57:50.585607 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585486 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c"} Jul 17 18:57:50.585607 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585514 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425"} Jul 17 18:57:50.585607 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:50.585546 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e"} Jul 17 18:57:51.429865 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.429842 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.523339 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523251 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523339 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523287 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cb2j\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523339 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523308 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523339 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523324 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523339 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523346 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523383 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523414 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523456 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523487 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523516 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523552 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523629 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.523735 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523690 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume\") pod \"d58aff2c-7396-4024-9e1b-2274dd8134c0\" (UID: \"d58aff2c-7396-4024-9e1b-2274dd8134c0\") " Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523738 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523796 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.523827 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.524007 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-main-db\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.524117 2584 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.524284 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.524203 2584 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d58aff2c-7396-4024-9e1b-2274dd8134c0-metrics-client-ca\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.526931 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.526879 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out" (OuterVolumeSpecName: "config-out") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 18:57:51.527150 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.527126 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j" (OuterVolumeSpecName: "kube-api-access-4cb2j") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "kube-api-access-4cb2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:51.527763 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.527605 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.527763 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.527722 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.527903 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.527791 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:51.528164 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.528135 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.528329 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.528311 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume" (OuterVolumeSpecName: "config-volume") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.528458 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.528438 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.531436 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.531409 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.538993 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.538969 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config" (OuterVolumeSpecName: "web-config") pod "d58aff2c-7396-4024-9e1b-2274dd8134c0" (UID: "d58aff2c-7396-4024-9e1b-2274dd8134c0"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:51.592183 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592153 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" exitCode=0 Jul 17 18:57:51.592183 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592181 2584 generic.go:356] "Generic (PLEG): container finished" podID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerID="6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" exitCode=0 Jul 17 18:57:51.592367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592239 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.592367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592236 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4"} Jul 17 18:57:51.592367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592319 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208"} Jul 17 18:57:51.592367 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592347 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"d58aff2c-7396-4024-9e1b-2274dd8134c0","Type":"ContainerDied","Data":"b8fe2b1577eb8bec213515927d940308fbfd8fce45c9b1f8162a3df093c32d72"} Jul 17 18:57:51.592499 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.592380 2584 scope.go:117] "RemoveContainer" containerID="9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" Jul 17 18:57:51.601001 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.600982 2584 scope.go:117] "RemoveContainer" containerID="66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" Jul 17 18:57:51.607664 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.607622 2584 scope.go:117] "RemoveContainer" containerID="6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" Jul 17 18:57:51.617159 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.616867 2584 scope.go:117] "RemoveContainer" containerID="6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" Jul 17 18:57:51.618491 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.618426 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:51.621118 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.621096 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:51.624561 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624548 2584 scope.go:117] "RemoveContainer" containerID="75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" Jul 17 18:57:51.624912 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624892 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624917 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-main-tls\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624928 2584 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-cluster-tls-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624938 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624948 2584 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-volume\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624955 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cb2j\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-kube-api-access-4cb2j\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.624969 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624965 2584 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.625191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624976 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d58aff2c-7396-4024-9e1b-2274dd8134c0-web-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.625191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624988 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d58aff2c-7396-4024-9e1b-2274dd8134c0-config-out\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.625191 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.624995 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d58aff2c-7396-4024-9e1b-2274dd8134c0-tls-assets\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:51.631364 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.631347 2584 scope.go:117] "RemoveContainer" containerID="b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" Jul 17 18:57:51.638374 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.638330 2584 scope.go:117] "RemoveContainer" containerID="f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8" Jul 17 18:57:51.645704 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.645687 2584 scope.go:117] "RemoveContainer" containerID="9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" Jul 17 18:57:51.645965 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.645948 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71\": container with ID starting with 9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71 not found: ID does not exist" containerID="9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" Jul 17 18:57:51.646013 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.645975 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71"} err="failed to get container status \"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71\": rpc error: code = NotFound desc = could not find container \"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71\": container with ID starting with 9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71 not found: ID does not exist" Jul 17 18:57:51.646013 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.645991 2584 scope.go:117] "RemoveContainer" containerID="66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" Jul 17 18:57:51.646219 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.646195 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4\": container with ID starting with 66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4 not found: ID does not exist" containerID="66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" Jul 17 18:57:51.646263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646223 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4"} err="failed to get container status \"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4\": rpc error: code = NotFound desc = could not find container \"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4\": container with ID starting with 66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4 not found: ID does not exist" Jul 17 18:57:51.646263 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646239 2584 scope.go:117] "RemoveContainer" containerID="6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" Jul 17 18:57:51.646500 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.646448 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c\": container with ID starting with 6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c not found: ID does not exist" containerID="6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" Jul 17 18:57:51.646500 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646483 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c"} err="failed to get container status \"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c\": rpc error: code = NotFound desc = could not find container \"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c\": container with ID starting with 6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c not found: ID does not exist" Jul 17 18:57:51.646856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646505 2584 scope.go:117] "RemoveContainer" containerID="6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" Jul 17 18:57:51.646947 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.646869 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208\": container with ID starting with 6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208 not found: ID does not exist" containerID="6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" Jul 17 18:57:51.646947 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646896 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208"} err="failed to get container status \"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208\": rpc error: code = NotFound desc = could not find container \"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208\": container with ID starting with 6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208 not found: ID does not exist" Jul 17 18:57:51.646947 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.646916 2584 scope.go:117] "RemoveContainer" containerID="75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" Jul 17 18:57:51.647439 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.647397 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425\": container with ID starting with 75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425 not found: ID does not exist" containerID="75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" Jul 17 18:57:51.647439 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.647427 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425"} err="failed to get container status \"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425\": rpc error: code = NotFound desc = could not find container \"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425\": container with ID starting with 75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425 not found: ID does not exist" Jul 17 18:57:51.647547 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.647458 2584 scope.go:117] "RemoveContainer" containerID="b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" Jul 17 18:57:51.647803 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.647782 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e\": container with ID starting with b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e not found: ID does not exist" containerID="b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" Jul 17 18:57:51.647913 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.647809 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e"} err="failed to get container status \"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e\": rpc error: code = NotFound desc = could not find container \"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e\": container with ID starting with b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e not found: ID does not exist" Jul 17 18:57:51.647913 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.647829 2584 scope.go:117] "RemoveContainer" containerID="f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8" Jul 17 18:57:51.648131 ip-10-0-133-230 kubenswrapper[2584]: E0717 18:57:51.648112 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8\": container with ID starting with f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8 not found: ID does not exist" containerID="f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8" Jul 17 18:57:51.648221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648138 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8"} err="failed to get container status \"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8\": rpc error: code = NotFound desc = could not find container \"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8\": container with ID starting with f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8 not found: ID does not exist" Jul 17 18:57:51.648221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648158 2584 scope.go:117] "RemoveContainer" containerID="9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71" Jul 17 18:57:51.648429 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648407 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71"} err="failed to get container status \"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71\": rpc error: code = NotFound desc = could not find container \"9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71\": container with ID starting with 9252a0e260777d68083cce6ffebcbfe1f586a0f83bc1b631fda62c02fecbdf71 not found: ID does not exist" Jul 17 18:57:51.648429 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648428 2584 scope.go:117] "RemoveContainer" containerID="66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4" Jul 17 18:57:51.648669 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648625 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4"} err="failed to get container status \"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4\": rpc error: code = NotFound desc = could not find container \"66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4\": container with ID starting with 66ed12568f6731abaa5861a144c63dd19dc40f106952134c1da4d569198276a4 not found: ID does not exist" Jul 17 18:57:51.648669 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648666 2584 scope.go:117] "RemoveContainer" containerID="6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c" Jul 17 18:57:51.648926 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648906 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c"} err="failed to get container status \"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c\": rpc error: code = NotFound desc = could not find container \"6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c\": container with ID starting with 6eadd0846146c9eb3325fda2b4b7b9ddb0ff9c5a81104b95c638c819edcee17c not found: ID does not exist" Jul 17 18:57:51.648926 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648926 2584 scope.go:117] "RemoveContainer" containerID="6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208" Jul 17 18:57:51.649048 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.648945 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:51.649152 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649126 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208"} err="failed to get container status \"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208\": rpc error: code = NotFound desc = could not find container \"6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208\": container with ID starting with 6d934a49599089533f9bbefe3cbae6293d75053af9564b102f43df5bee193208 not found: ID does not exist" Jul 17 18:57:51.649152 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649149 2584 scope.go:117] "RemoveContainer" containerID="75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425" Jul 17 18:57:51.649299 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649273 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="prom-label-proxy" Jul 17 18:57:51.649299 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649291 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="prom-label-proxy" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649304 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649310 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649320 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="61ab76a6-14c5-40ae-8d47-1b26132d86da" containerName="console" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649327 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="61ab76a6-14c5-40ae-8d47-1b26132d86da" containerName="console" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649334 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-web" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649339 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-web" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649347 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-metric" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649352 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-metric" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649346 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425"} err="failed to get container status \"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425\": rpc error: code = NotFound desc = could not find container \"75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425\": container with ID starting with 75c6007ae9f72175f40f38733821e3dc1756c34da25a00bcfb86299cdd637425 not found: ID does not exist" Jul 17 18:57:51.649382 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649369 2584 scope.go:117] "RemoveContainer" containerID="b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649359 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="config-reloader" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649416 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="config-reloader" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649431 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="init-config-reloader" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649437 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="init-config-reloader" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649447 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649455 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649467 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="alertmanager" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649473 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="alertmanager" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649479 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="27c408de-184a-406d-bf6b-f13db23c37d2" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649484 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="27c408de-184a-406d-bf6b-f13db23c37d2" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649561 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="prom-label-proxy" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649571 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="61ab76a6-14c5-40ae-8d47-1b26132d86da" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649577 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649584 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="27c408de-184a-406d-bf6b-f13db23c37d2" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649590 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-web" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649601 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="de49ac0f-7b0f-413f-9fb5-9aa67c316d50" containerName="console" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649609 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="alertmanager" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649617 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="config-reloader" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649624 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" containerName="kube-rbac-proxy-metric" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649613 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e"} err="failed to get container status \"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e\": rpc error: code = NotFound desc = could not find container \"b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e\": container with ID starting with b41f7133a0c7c21d11d8911a7ead45da183ff19f9c42cbdc8c37e0bb7179898e not found: ID does not exist" Jul 17 18:57:51.649788 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649668 2584 scope.go:117] "RemoveContainer" containerID="f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8" Jul 17 18:57:51.650618 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.649892 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8"} err="failed to get container status \"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8\": rpc error: code = NotFound desc = could not find container \"f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8\": container with ID starting with f1cd22c38dc42417138051f8b326b485bc3dc3c5d0522eb54aede7f2a58d59e8 not found: ID does not exist" Jul 17 18:57:51.654729 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.654711 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.657941 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.657918 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 17 18:57:51.658026 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.657941 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 17 18:57:51.658026 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.657967 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 17 18:57:51.658026 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.657981 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 17 18:57:51.658026 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.657967 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 17 18:57:51.658026 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.658023 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 17 18:57:51.658426 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.658412 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 17 18:57:51.658484 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.658417 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-nk9rk\"" Jul 17 18:57:51.658484 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.658445 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 17 18:57:51.662588 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.662568 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 17 18:57:51.672825 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.671730 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:51.725654 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725607 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.725801 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725744 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.725871 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725797 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.725871 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725827 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.725965 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725908 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726007 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.725975 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726049 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726027 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726139 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726124 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-out\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726155 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726241 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726188 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-web-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726241 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726213 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ls95p\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-kube-api-access-ls95p\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726323 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726262 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.726323 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.726292 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827483 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827395 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-out\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827483 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827434 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827483 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827459 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-web-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827483 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827479 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ls95p\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-kube-api-access-ls95p\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827519 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827549 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827582 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827623 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827683 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827710 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827752 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827779 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.827815 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.827808 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.828961 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.828933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.829109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.829079 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.829555 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.829530 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831013 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.830988 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831104 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831058 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831288 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831271 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-out\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831547 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-tls-assets\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831574 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831574 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-web-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831767 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831586 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831767 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831584 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-config-volume\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.831767 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.831675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.832453 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.832436 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.836025 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.836007 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ls95p\" (UniqueName: \"kubernetes.io/projected/703b7e90-3d53-4ceb-bb0b-c2a35e3876ac-kube-api-access-ls95p\") pod \"alertmanager-main-0\" (UID: \"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac\") " pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:51.964172 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:51.964139 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 17 18:57:52.101942 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:52.101863 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 17 18:57:52.104615 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:57:52.104586 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod703b7e90_3d53_4ceb_bb0b_c2a35e3876ac.slice/crio-0953eab666e7549e0e3e51c575dde14868645a712d67a02bcb51fa930c11fd12 WatchSource:0}: Error finding container 0953eab666e7549e0e3e51c575dde14868645a712d67a02bcb51fa930c11fd12: Status 404 returned error can't find the container with id 0953eab666e7549e0e3e51c575dde14868645a712d67a02bcb51fa930c11fd12 Jul 17 18:57:52.599305 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:52.599275 2584 generic.go:356] "Generic (PLEG): container finished" podID="703b7e90-3d53-4ceb-bb0b-c2a35e3876ac" containerID="0c4397147d37c2431170a35bbdec727c1bf4f736076782bcd0c649133750c73a" exitCode=0 Jul 17 18:57:52.599714 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:52.599325 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerDied","Data":"0c4397147d37c2431170a35bbdec727c1bf4f736076782bcd0c649133750c73a"} Jul 17 18:57:52.599714 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:52.599372 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"0953eab666e7549e0e3e51c575dde14868645a712d67a02bcb51fa930c11fd12"} Jul 17 18:57:52.904990 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:52.904956 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d58aff2c-7396-4024-9e1b-2274dd8134c0" path="/var/lib/kubelet/pods/d58aff2c-7396-4024-9e1b-2274dd8134c0/volumes" Jul 17 18:57:53.607128 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607094 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"49c5aed99d4b3e53bae5a3ce83502a2f4c17997612c0ec794c9d9546f2645417"} Jul 17 18:57:53.607529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607158 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"6cdd6abc6ad2b6c6050d040e7a0cc744b1355df03e27027f402f5af9aa959583"} Jul 17 18:57:53.607529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607185 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"c272a29d0fcbdc3072d4ca18cef55d3081e2cdbd90aa0ce8ba60b362ece876e7"} Jul 17 18:57:53.607529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607209 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"5efadbd54c41d179fdb2ddb93cc4246ef1e4f88133ffe036c79fc50b7705c208"} Jul 17 18:57:53.607529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607232 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"52a9fbbf0ffa625b8a315ff4f30369753ba98fb47e0c525c7f9777f6e3467577"} Jul 17 18:57:53.607529 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.607254 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"703b7e90-3d53-4ceb-bb0b-c2a35e3876ac","Type":"ContainerStarted","Data":"f329ce4c6c34680a063be9ea45ef20fbfee6275ff0117bd803643f12bfd01f8e"} Jul 17 18:57:53.638309 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:53.638256 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.619038508 podStartE2EDuration="2.638239094s" totalImagesPullingTime="0s" totalInitContainerRuntime="19.200586ms" isStatefulPod=false podCreationTimestamp="2026-07-17 18:57:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:57:53.637044374 +0000 UTC m=+155.288424831" watchObservedRunningTime="2026-07-17 18:57:53.638239094 +0000 UTC m=+155.289619583" Jul 17 18:57:54.416830 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.416797 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:54.417128 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417095 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="prometheus" containerID="cri-o://223b9ad56493e890203681a7fb9842b1cbd0305bfa9a4b719c79213e322cca2f" gracePeriod=600 Jul 17 18:57:54.417214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417125 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy" containerID="cri-o://97c1022b7ca98ecd5958b26997c7525184abf145ae0fcaa3661a3ad081d3de0e" gracePeriod=600 Jul 17 18:57:54.417214 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417151 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-thanos" containerID="cri-o://1393306fbac778da18ff37fa939268d476760296a786cf1fec30d3ef72305445" gracePeriod=600 Jul 17 18:57:54.417350 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417214 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="config-reloader" containerID="cri-o://974549d1e7ccabe69504d4874d8e3e7d88ef9797e6ffbc3ee4893d4ea17ea589" gracePeriod=600 Jul 17 18:57:54.417350 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417169 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-web" containerID="cri-o://634ac45890b6d698ba57d965b7daf57bbcdaa4324f2ded8a5aa6e4cf6065d92e" gracePeriod=600 Jul 17 18:57:54.417350 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.417169 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="thanos-sidecar" containerID="cri-o://6e93eefab70e7f1fd4864a2d249d55420763b21e665333b5c1bc01fd0b86fc8d" gracePeriod=600 Jul 17 18:57:54.618278 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618247 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="1393306fbac778da18ff37fa939268d476760296a786cf1fec30d3ef72305445" exitCode=0 Jul 17 18:57:54.618278 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618274 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="97c1022b7ca98ecd5958b26997c7525184abf145ae0fcaa3661a3ad081d3de0e" exitCode=0 Jul 17 18:57:54.618278 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618284 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="634ac45890b6d698ba57d965b7daf57bbcdaa4324f2ded8a5aa6e4cf6065d92e" exitCode=0 Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618292 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="6e93eefab70e7f1fd4864a2d249d55420763b21e665333b5c1bc01fd0b86fc8d" exitCode=0 Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618301 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="974549d1e7ccabe69504d4874d8e3e7d88ef9797e6ffbc3ee4893d4ea17ea589" exitCode=0 Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618309 2584 generic.go:356] "Generic (PLEG): container finished" podID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerID="223b9ad56493e890203681a7fb9842b1cbd0305bfa9a4b719c79213e322cca2f" exitCode=0 Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618322 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"1393306fbac778da18ff37fa939268d476760296a786cf1fec30d3ef72305445"} Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618417 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"97c1022b7ca98ecd5958b26997c7525184abf145ae0fcaa3661a3ad081d3de0e"} Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618460 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"634ac45890b6d698ba57d965b7daf57bbcdaa4324f2ded8a5aa6e4cf6065d92e"} Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618499 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"6e93eefab70e7f1fd4864a2d249d55420763b21e665333b5c1bc01fd0b86fc8d"} Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618536 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"974549d1e7ccabe69504d4874d8e3e7d88ef9797e6ffbc3ee4893d4ea17ea589"} Jul 17 18:57:54.618710 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.618570 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"223b9ad56493e890203681a7fb9842b1cbd0305bfa9a4b719c79213e322cca2f"} Jul 17 18:57:54.675224 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.675151 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:54.755221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755191 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755224 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755459 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755242 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755459 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755266 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755459 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755282 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755463 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755496 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755526 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755553 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755598 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.755626 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755611 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755633 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755671 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755702 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755738 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b6xrf\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755764 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755798 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755824 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755864 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755889 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.755912 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756109 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.756031 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle\") pod \"1273390c-3f28-4ac3-952f-6c8b1e068ead\" (UID: \"1273390c-3f28-4ac3-952f-6c8b1e068ead\") " Jul 17 18:57:54.756628 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.756403 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.756628 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.756424 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.756756 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.756635 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.757210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.757176 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.757608 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.757585 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.758498 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.758409 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.758498 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.758433 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.758498 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.758471 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.758723 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.758504 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config" (OuterVolumeSpecName: "config") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.758723 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.758551 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 18:57:54.759392 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.759356 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 18:57:54.760709 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.760682 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:54.761210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761066 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out" (OuterVolumeSpecName: "config-out") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 18:57:54.761582 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761460 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.761582 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761543 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.761582 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761569 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.761813 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761787 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.761813 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.761790 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf" (OuterVolumeSpecName: "kube-api-access-b6xrf") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "kube-api-access-b6xrf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 18:57:54.762219 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.762197 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.770210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.770183 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config" (OuterVolumeSpecName: "web-config") pod "1273390c-3f28-4ac3-952f-6c8b1e068ead" (UID: "1273390c-3f28-4ac3-952f-6c8b1e068ead"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 18:57:54.856982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.856938 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.856982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.856976 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-tls\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.856982 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.856990 2584 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-kube-rbac-proxy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857002 2584 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-config-out\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857015 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857027 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-metrics-client-ca\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857041 2584 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-thanos-prometheus-http-client-file\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857053 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b6xrf\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-kube-api-access-b6xrf\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857066 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857078 2584 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-grpc-tls\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857089 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-trusted-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857101 2584 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857114 2584 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1273390c-3f28-4ac3-952f-6c8b1e068ead-prometheus-k8s-db\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857125 2584 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1273390c-3f28-4ac3-952f-6c8b1e068ead-tls-assets\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857137 2584 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1273390c-3f28-4ac3-952f-6c8b1e068ead-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857148 2584 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-web-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857160 2584 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-secret-metrics-client-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:54.857249 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:54.857172 2584 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1273390c-3f28-4ac3-952f-6c8b1e068ead-config\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 18:57:55.626035 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.626011 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.626479 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.626004 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1273390c-3f28-4ac3-952f-6c8b1e068ead","Type":"ContainerDied","Data":"a2d1c92cc00553268b616e08eea00c236b2b5a17a2790656fb6f1ec4c54d8933"} Jul 17 18:57:55.626479 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.626161 2584 scope.go:117] "RemoveContainer" containerID="1393306fbac778da18ff37fa939268d476760296a786cf1fec30d3ef72305445" Jul 17 18:57:55.634635 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.634619 2584 scope.go:117] "RemoveContainer" containerID="97c1022b7ca98ecd5958b26997c7525184abf145ae0fcaa3661a3ad081d3de0e" Jul 17 18:57:55.643065 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.643038 2584 scope.go:117] "RemoveContainer" containerID="634ac45890b6d698ba57d965b7daf57bbcdaa4324f2ded8a5aa6e4cf6065d92e" Jul 17 18:57:55.651060 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.651043 2584 scope.go:117] "RemoveContainer" containerID="6e93eefab70e7f1fd4864a2d249d55420763b21e665333b5c1bc01fd0b86fc8d" Jul 17 18:57:55.651659 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.651627 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:55.654876 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.654855 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:55.658563 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.658546 2584 scope.go:117] "RemoveContainer" containerID="974549d1e7ccabe69504d4874d8e3e7d88ef9797e6ffbc3ee4893d4ea17ea589" Jul 17 18:57:55.665350 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.665331 2584 scope.go:117] "RemoveContainer" containerID="223b9ad56493e890203681a7fb9842b1cbd0305bfa9a4b719c79213e322cca2f" Jul 17 18:57:55.672516 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.672498 2584 scope.go:117] "RemoveContainer" containerID="17a9418ee0fd08cfc56a9dc7982a5422cdba1981a6d4c74237033c3b97de6960" Jul 17 18:57:55.681316 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681296 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:55.681588 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681577 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="thanos-sidecar" Jul 17 18:57:55.681629 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681591 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="thanos-sidecar" Jul 17 18:57:55.681629 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681603 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-web" Jul 17 18:57:55.681629 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681609 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-web" Jul 17 18:57:55.681629 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681624 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-thanos" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681632 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-thanos" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681669 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="init-config-reloader" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681675 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="init-config-reloader" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681689 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="config-reloader" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681694 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="config-reloader" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681701 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="prometheus" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681706 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="prometheus" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681712 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681717 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy" Jul 17 18:57:55.681762 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681762 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-web" Jul 17 18:57:55.682051 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681776 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="config-reloader" Jul 17 18:57:55.682051 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681787 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="prometheus" Jul 17 18:57:55.682051 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681793 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy-thanos" Jul 17 18:57:55.682051 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681799 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="thanos-sidecar" Jul 17 18:57:55.682051 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.681803 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" containerName="kube-rbac-proxy" Jul 17 18:57:55.687325 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.687283 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.691041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.690692 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 17 18:57:55.691041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.690815 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-ct8sz\"" Jul 17 18:57:55.691041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.690907 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 17 18:57:55.691041 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.690984 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 17 18:57:55.691789 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.691724 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 17 18:57:55.692159 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.692081 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 17 18:57:55.692338 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.692323 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 17 18:57:55.692691 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.692375 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 17 18:57:55.692947 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.692928 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 17 18:57:55.693213 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.693185 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 17 18:57:55.693474 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.693456 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 17 18:57:55.693686 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.693663 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 17 18:57:55.693961 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.693946 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 17 18:57:55.694205 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.694169 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-buro0hbftskms\"" Jul 17 18:57:55.698447 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.698427 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 17 18:57:55.699259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.699239 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 17 18:57:55.702050 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.702032 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:55.765690 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765631 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765704 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765745 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765856 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765779 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config-out\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765851 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765892 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765909 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2gjv\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-kube-api-access-k2gjv\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765929 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-web-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765962 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765979 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.765996 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.765997 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766064 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766101 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766131 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766178 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766214 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766246 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766269 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766296 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.766320 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.766313 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866767 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866733 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config-out\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866774 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866799 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866816 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2gjv\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-kube-api-access-k2gjv\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866843 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-web-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866872 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.866904 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866897 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.866987 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867038 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867067 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867095 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867127 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867162 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867192 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867210 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867215 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867310 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867345 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867374 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.867795 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.867607 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.869228 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.868252 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.869228 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.868823 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.869689 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.869636 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870027 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870089 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870127 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config-out\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870221 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870131 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870671 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870601 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.870671 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870615 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.871019 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.870980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-web-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.871259 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.871234 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.871460 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.871419 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.871541 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.871454 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.872971 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.872942 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-config\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.873058 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.873011 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.873125 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.873095 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.873359 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.873338 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.873950 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.873933 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:55.878691 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:55.878673 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2gjv\" (UniqueName: \"kubernetes.io/projected/d41ad0f7-bfed-4be6-af63-4c14d21ddc18-kube-api-access-k2gjv\") pod \"prometheus-k8s-0\" (UID: \"d41ad0f7-bfed-4be6-af63-4c14d21ddc18\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:56.003866 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.003830 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:57:56.208999 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.208971 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 17 18:57:56.210690 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:57:56.210636 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd41ad0f7_bfed_4be6_af63_4c14d21ddc18.slice/crio-cbea2a7f5f1406c74128139b731600ef48a95328d585417d4d97337ee89cffc6 WatchSource:0}: Error finding container cbea2a7f5f1406c74128139b731600ef48a95328d585417d4d97337ee89cffc6: Status 404 returned error can't find the container with id cbea2a7f5f1406c74128139b731600ef48a95328d585417d4d97337ee89cffc6 Jul 17 18:57:56.631192 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.631107 2584 generic.go:356] "Generic (PLEG): container finished" podID="d41ad0f7-bfed-4be6-af63-4c14d21ddc18" containerID="cb2cbc8281337b69f741d22ed8dac07e47e5b95c2529ce0c7fe1ebfbbe250977" exitCode=0 Jul 17 18:57:56.631602 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.631189 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerDied","Data":"cb2cbc8281337b69f741d22ed8dac07e47e5b95c2529ce0c7fe1ebfbbe250977"} Jul 17 18:57:56.631602 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.631256 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"cbea2a7f5f1406c74128139b731600ef48a95328d585417d4d97337ee89cffc6"} Jul 17 18:57:56.907078 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:56.907048 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1273390c-3f28-4ac3-952f-6c8b1e068ead" path="/var/lib/kubelet/pods/1273390c-3f28-4ac3-952f-6c8b1e068ead/volumes" Jul 17 18:57:57.644596 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644550 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"73feb587fcd20d110e2e204c65318c2dbec45a6408935daefdf8ee2a4b92e334"} Jul 17 18:57:57.644962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644661 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"23d4090cdd52990dbaccafd92f7d593667b92a48518c53af17007d26bf97a727"} Jul 17 18:57:57.644962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644689 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"b1fe7805a6e022fcb844bd7f681df8b5d775791fecd95227def13b3ee574ecd1"} Jul 17 18:57:57.644962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644714 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"f26c459cb4c4298be477d94658e38387c1eca6fc5bfbd85735b0aa3859db0e09"} Jul 17 18:57:57.644962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644736 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"47c4ae1c4eaf158ec02fb4099b4e392214e475b11458f2f3c2cbf77b9de96f78"} Jul 17 18:57:57.644962 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.644758 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"d41ad0f7-bfed-4be6-af63-4c14d21ddc18","Type":"ContainerStarted","Data":"d105dcab20980699fa5733da4bbc76b6eb5204e0a469271c08429ddda78b8fff"} Jul 17 18:57:57.677392 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:57:57.677345 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.65641266 podStartE2EDuration="2.677328235s" totalImagesPullingTime="0s" totalInitContainerRuntime="20.915575ms" isStatefulPod=false podCreationTimestamp="2026-07-17 18:57:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:57:57.674930217 +0000 UTC m=+159.326310686" watchObservedRunningTime="2026-07-17 18:57:57.677328235 +0000 UTC m=+159.328708689" Jul 17 18:58:01.004332 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:58:01.004265 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:58:56.004816 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:58:56.004781 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:58:56.021075 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:58:56.021045 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:58:56.888812 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:58:56.888786 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 17 18:59:42.695980 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.695903 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft"] Jul 17 18:59:42.699072 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.699056 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.701879 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.701838 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 17 18:59:42.702024 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.702006 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 17 18:59:42.703146 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.703132 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-cgq58\"" Jul 17 18:59:42.710409 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.710387 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft"] Jul 17 18:59:42.712913 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.712889 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fmr2n\" (UniqueName: \"kubernetes.io/projected/4b508fa1-c46b-4ace-970c-94ff9d045075-kube-api-access-fmr2n\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.713014 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.712932 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b508fa1-c46b-4ace-970c-94ff9d045075-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.814097 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.814067 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fmr2n\" (UniqueName: \"kubernetes.io/projected/4b508fa1-c46b-4ace-970c-94ff9d045075-kube-api-access-fmr2n\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.814282 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.814119 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b508fa1-c46b-4ace-970c-94ff9d045075-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.814438 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.814421 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b508fa1-c46b-4ace-970c-94ff9d045075-tmp\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:42.828752 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:42.828717 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fmr2n\" (UniqueName: \"kubernetes.io/projected/4b508fa1-c46b-4ace-970c-94ff9d045075-kube-api-access-fmr2n\") pod \"openshift-lws-operator-fd8ccff4c-9t6ft\" (UID: \"4b508fa1-c46b-4ace-970c-94ff9d045075\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:43.026781 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:43.026747 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" Jul 17 18:59:43.157860 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:43.157836 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft"] Jul 17 18:59:43.160611 ip-10-0-133-230 kubenswrapper[2584]: W0717 18:59:43.160584 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b508fa1_c46b_4ace_970c_94ff9d045075.slice/crio-59bbe4c11b67937ff87c31d325e817bc8f91e7f2924f7d9a9516c9d06f03ccfb WatchSource:0}: Error finding container 59bbe4c11b67937ff87c31d325e817bc8f91e7f2924f7d9a9516c9d06f03ccfb: Status 404 returned error can't find the container with id 59bbe4c11b67937ff87c31d325e817bc8f91e7f2924f7d9a9516c9d06f03ccfb Jul 17 18:59:44.050712 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:44.050678 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" event={"ID":"4b508fa1-c46b-4ace-970c-94ff9d045075","Type":"ContainerStarted","Data":"59bbe4c11b67937ff87c31d325e817bc8f91e7f2924f7d9a9516c9d06f03ccfb"} Jul 17 18:59:55.095017 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:55.094928 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" event={"ID":"4b508fa1-c46b-4ace-970c-94ff9d045075","Type":"ContainerStarted","Data":"159e3a849bfe9540125f743742e184dafdbb17ccdb064a978edc858ee5d2e44a"} Jul 17 18:59:55.113594 ip-10-0-133-230 kubenswrapper[2584]: I0717 18:59:55.113548 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-9t6ft" podStartSLOduration=1.43740673 podStartE2EDuration="13.113531002s" totalImagesPullingTime="11.676124272s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 18:59:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 18:59:55.111765555 +0000 UTC m=+276.763146010" watchObservedRunningTime="2026-07-17 18:59:55.113531002 +0000 UTC m=+276.764911457" Jul 17 19:00:13.021234 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.021192 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h"] Jul 17 19:00:13.025075 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.025052 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.028294 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.028273 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 17 19:00:13.028732 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.028714 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 17 19:00:13.029045 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.029007 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 17 19:00:13.029045 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.029039 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 17 19:00:13.029292 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.029278 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-p5gcz\"" Jul 17 19:00:13.046456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.046433 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h"] Jul 17 19:00:13.166599 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.166568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-manifests\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.166599 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.166602 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qb6xx\" (UniqueName: \"kubernetes.io/projected/559b0b1a-7216-46a5-810f-553605103653-kube-api-access-qb6xx\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.166855 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.166633 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-tmp\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.166855 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.166708 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-webhook-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.166855 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.166767 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267395 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267365 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267442 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-manifests\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267464 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qb6xx\" (UniqueName: \"kubernetes.io/projected/559b0b1a-7216-46a5-810f-553605103653-kube-api-access-qb6xx\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267491 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-tmp\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267508 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-webhook-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.267941 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.267906 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-manifests\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.270003 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.269981 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/559b0b1a-7216-46a5-810f-553605103653-tmp\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.270236 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.270215 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-webhook-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.270281 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.270253 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/559b0b1a-7216-46a5-810f-553605103653-apiservice-cert\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.276452 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.276401 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qb6xx\" (UniqueName: \"kubernetes.io/projected/559b0b1a-7216-46a5-810f-553605103653-kube-api-access-qb6xx\") pod \"opendatahub-operator-controller-manager-b55dc8c5f-9sv6h\" (UID: \"559b0b1a-7216-46a5-810f-553605103653\") " pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.335596 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.335563 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:13.482136 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:13.482111 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h"] Jul 17 19:00:13.484720 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:00:13.484690 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod559b0b1a_7216_46a5_810f_553605103653.slice/crio-745de8d0fab233d9da16795a3dff06e79395e19b6a8dff46cbb13c567d5c2ebd WatchSource:0}: Error finding container 745de8d0fab233d9da16795a3dff06e79395e19b6a8dff46cbb13c567d5c2ebd: Status 404 returned error can't find the container with id 745de8d0fab233d9da16795a3dff06e79395e19b6a8dff46cbb13c567d5c2ebd Jul 17 19:00:14.174681 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:14.174628 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" event={"ID":"559b0b1a-7216-46a5-810f-553605103653","Type":"ContainerStarted","Data":"745de8d0fab233d9da16795a3dff06e79395e19b6a8dff46cbb13c567d5c2ebd"} Jul 17 19:00:18.195257 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:18.195218 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" event={"ID":"559b0b1a-7216-46a5-810f-553605103653","Type":"ContainerStarted","Data":"548720c01abefdc1887cd166b652d043a79e3431037dc7927d1650ff0fe00713"} Jul 17 19:00:18.737200 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:18.737175 2584 kubelet.go:1648] "Image garbage collection succeeded" Jul 17 19:00:19.201405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:19.201375 2584 generic.go:356] "Generic (PLEG): container finished" podID="559b0b1a-7216-46a5-810f-553605103653" containerID="548720c01abefdc1887cd166b652d043a79e3431037dc7927d1650ff0fe00713" exitCode=0 Jul 17 19:00:19.203793 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:19.201464 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" event={"ID":"559b0b1a-7216-46a5-810f-553605103653","Type":"ContainerDied","Data":"548720c01abefdc1887cd166b652d043a79e3431037dc7927d1650ff0fe00713"} Jul 17 19:00:20.208044 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:20.208013 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" event={"ID":"559b0b1a-7216-46a5-810f-553605103653","Type":"ContainerStarted","Data":"779822466ce889a8b6b9316697adac18aa63c2cd703e03f84dcad22f35a5d9bb"} Jul 17 19:00:20.208421 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:20.208156 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:20.234616 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:20.234563 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" podStartSLOduration=3.010757451 podStartE2EDuration="8.234545481s" totalImagesPullingTime="4.21022369s" totalInitContainerRuntime="1.01356434s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:00:12 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:00:20.232102309 +0000 UTC m=+301.883482763" watchObservedRunningTime="2026-07-17 19:00:20.234545481 +0000 UTC m=+301.885925937" Jul 17 19:00:30.808129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.808092 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms"] Jul 17 19:00:30.821548 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.821515 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms"] Jul 17 19:00:30.821737 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.821623 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:30.825950 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.825921 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 17 19:00:30.826102 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.825922 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 17 19:00:30.826102 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.825921 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ksw94\"" Jul 17 19:00:30.826102 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.826093 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 17 19:00:30.926686 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.926628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:30.926883 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.926698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msnml\" (UniqueName: \"kubernetes.io/projected/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-kube-api-access-msnml\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:30.926883 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.926755 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-manager-config\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:30.926883 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:30.926829 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-metrics-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.028053 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.028021 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-metrics-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.028216 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.028093 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.028216 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.028122 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msnml\" (UniqueName: \"kubernetes.io/projected/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-kube-api-access-msnml\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.028396 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.028242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-manager-config\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.029023 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.028994 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-manager-config\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.030613 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.030591 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-metrics-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.030734 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.030719 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-cert\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.037043 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.037021 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msnml\" (UniqueName: \"kubernetes.io/projected/4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3-kube-api-access-msnml\") pod \"lws-controller-manager-64d7f9f4cf-d2vms\" (UID: \"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3\") " pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.132170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.132090 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:31.214314 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.214227 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-b55dc8c5f-9sv6h" Jul 17 19:00:31.261301 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.261277 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms"] Jul 17 19:00:31.263361 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:00:31.263334 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4944bffe_e2d1_4dee_93d2_bcfb5f64a5a3.slice/crio-0a99d591e629417edbb6a35ddab092644a00102d5343c7d6c162d6df4da87ecf WatchSource:0}: Error finding container 0a99d591e629417edbb6a35ddab092644a00102d5343c7d6c162d6df4da87ecf: Status 404 returned error can't find the container with id 0a99d591e629417edbb6a35ddab092644a00102d5343c7d6c162d6df4da87ecf Jul 17 19:00:31.265741 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:31.265720 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 19:00:32.259514 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:32.259473 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" event={"ID":"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3","Type":"ContainerStarted","Data":"0a99d591e629417edbb6a35ddab092644a00102d5343c7d6c162d6df4da87ecf"} Jul 17 19:00:36.280035 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:36.279997 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" event={"ID":"4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3","Type":"ContainerStarted","Data":"625f6ae54c78af910b11cdb590964585771a1e6d0536ddc8f73372337e81787f"} Jul 17 19:00:36.280415 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:36.280079 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:36.298846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:36.298793 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" podStartSLOduration=2.143728292 podStartE2EDuration="6.298778189s" totalImagesPullingTime="4.155049897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:00:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:00:36.297243264 +0000 UTC m=+317.948623752" watchObservedRunningTime="2026-07-17 19:00:36.298778189 +0000 UTC m=+317.950158643" Jul 17 19:00:39.050607 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.050573 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq"] Jul 17 19:00:39.052934 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.052898 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.056985 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.056956 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 17 19:00:39.056985 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.056981 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 17 19:00:39.057162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.056995 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 17 19:00:39.057162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.057038 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 17 19:00:39.057162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.056971 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-685nt\"" Jul 17 19:00:39.065704 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.065679 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq"] Jul 17 19:00:39.104180 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.104141 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tmp\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.104180 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.104182 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tls-certs\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.104382 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.104252 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hfx9j\" (UniqueName: \"kubernetes.io/projected/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-kube-api-access-hfx9j\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.205090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.205052 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tmp\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.205090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.205092 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tls-certs\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.205300 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.205178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hfx9j\" (UniqueName: \"kubernetes.io/projected/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-kube-api-access-hfx9j\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.207542 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.207511 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tmp\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.207846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.207824 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-tls-certs\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.213983 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.213952 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hfx9j\" (UniqueName: \"kubernetes.io/projected/d0291cde-e2ae-4207-b9b1-5b61a0eca94f-kube-api-access-hfx9j\") pod \"kube-auth-proxy-ddf547cfd-mzwlq\" (UID: \"d0291cde-e2ae-4207-b9b1-5b61a0eca94f\") " pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.362767 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.362679 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" Jul 17 19:00:39.496003 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:39.495979 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq"] Jul 17 19:00:39.498453 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:00:39.498426 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0291cde_e2ae_4207_b9b1_5b61a0eca94f.slice/crio-68097cbab304d962dcb160d4fbed8c8153dfedcc53d7f4d752973b9580b44789 WatchSource:0}: Error finding container 68097cbab304d962dcb160d4fbed8c8153dfedcc53d7f4d752973b9580b44789: Status 404 returned error can't find the container with id 68097cbab304d962dcb160d4fbed8c8153dfedcc53d7f4d752973b9580b44789 Jul 17 19:00:40.299199 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:40.299163 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" event={"ID":"d0291cde-e2ae-4207-b9b1-5b61a0eca94f","Type":"ContainerStarted","Data":"68097cbab304d962dcb160d4fbed8c8153dfedcc53d7f4d752973b9580b44789"} Jul 17 19:00:42.312514 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:42.312479 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" event={"ID":"d0291cde-e2ae-4207-b9b1-5b61a0eca94f","Type":"ContainerStarted","Data":"1a5dccdb94894a2404c687f37b0ab57f0416b565d6ed354d3229b35b91264486"} Jul 17 19:00:42.337192 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:42.337127 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-ddf547cfd-mzwlq" podStartSLOduration=0.718336819 podStartE2EDuration="3.337104269s" totalImagesPullingTime="2.61876745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:00:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:00:42.336861263 +0000 UTC m=+323.988241739" watchObservedRunningTime="2026-07-17 19:00:42.337104269 +0000 UTC m=+323.988484725" Jul 17 19:00:46.451025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.450983 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc"] Jul 17 19:00:46.454132 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.454109 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.463365 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.463344 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-8whvb\"" Jul 17 19:00:46.463482 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.463454 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 17 19:00:46.488233 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.488207 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc"] Jul 17 19:00:46.575283 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575249 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddbg4\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-kube-api-access-ddbg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575283 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575288 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575379 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575406 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575426 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575441 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575502 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280bdaa2-bcdf-42e6-a181-16da6358990d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575535 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.575687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.575603 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676332 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676293 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676332 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676333 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676539 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676360 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676579 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676536 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676579 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676575 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280bdaa2-bcdf-42e6-a181-16da6358990d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676720 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676605 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676720 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676671 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676816 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676717 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddbg4\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-kube-api-access-ddbg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.676816 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676755 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.677034 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676914 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.677034 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.676979 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-data\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.677183 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.677053 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.677183 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.677076 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.677329 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.677308 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/280bdaa2-bcdf-42e6-a181-16da6358990d-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.678957 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.678936 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.679106 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.679089 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.686813 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.686787 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-istio-token\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.687560 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.687541 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddbg4\" (UniqueName: \"kubernetes.io/projected/280bdaa2-bcdf-42e6-a181-16da6358990d-kube-api-access-ddbg4\") pod \"data-science-gateway-data-science-gateway-class-6576859cf4mrppc\" (UID: \"280bdaa2-bcdf-42e6-a181-16da6358990d\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.763671 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.763569 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:46.916156 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:46.916131 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc"] Jul 17 19:00:46.917298 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:00:46.917273 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280bdaa2_bcdf_42e6_a181_16da6358990d.slice/crio-1cc0bd07252d6f40ba583ebf32510ebd5287cc660de042ce538373d2aacdd085 WatchSource:0}: Error finding container 1cc0bd07252d6f40ba583ebf32510ebd5287cc660de042ce538373d2aacdd085: Status 404 returned error can't find the container with id 1cc0bd07252d6f40ba583ebf32510ebd5287cc660de042ce538373d2aacdd085 Jul 17 19:00:47.287059 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:47.287031 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-64d7f9f4cf-d2vms" Jul 17 19:00:47.335588 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:47.335554 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" event={"ID":"280bdaa2-bcdf-42e6-a181-16da6358990d","Type":"ContainerStarted","Data":"1cc0bd07252d6f40ba583ebf32510ebd5287cc660de042ce538373d2aacdd085"} Jul 17 19:00:49.839380 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:49.839333 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:00:49.839810 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:49.839431 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:00:49.839810 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:49.839475 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:00:50.352386 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:50.352352 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" event={"ID":"280bdaa2-bcdf-42e6-a181-16da6358990d","Type":"ContainerStarted","Data":"66705e2e35778823698bcfd285aabf12a2a6fdf22edf20768b8bb3db8d9cdf4d"} Jul 17 19:00:50.393590 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:50.393533 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" podStartSLOduration=1.474003522 podStartE2EDuration="4.393517351s" totalImagesPullingTime="2.919513829s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:00:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:00:50.391986675 +0000 UTC m=+332.043367151" watchObservedRunningTime="2026-07-17 19:00:50.393517351 +0000 UTC m=+332.044897805" Jul 17 19:00:50.764350 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:50.764315 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:50.768829 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:50.768800 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:51.357167 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:51.357134 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:00:51.358280 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:00:51.358258 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-6576859cf4mrppc" Jul 17 19:01:59.110175 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.110137 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-8cc7cdf69-57snr"] Jul 17 19:01:59.113653 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.113627 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.116918 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.116890 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 17 19:01:59.117029 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.116939 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 17 19:01:59.118005 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.117985 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 17 19:01:59.118233 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.118211 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 17 19:01:59.118341 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.118286 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-bdvh2\"" Jul 17 19:01:59.118460 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.118442 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 17 19:01:59.118524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.118507 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 17 19:01:59.119596 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.119574 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 17 19:01:59.122499 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.122478 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 17 19:01:59.126134 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.126114 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cc7cdf69-57snr"] Jul 17 19:01:59.232796 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232762 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-trusted-ca-bundle\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.232954 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232838 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqzdh\" (UniqueName: \"kubernetes.io/projected/9640041c-9183-48db-b0c6-dd0cc9c192e2-kube-api-access-wqzdh\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.232954 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232918 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-oauth-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.232954 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.233062 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232963 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.233062 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.232998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-service-ca\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.233062 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.233029 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-oauth-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.333753 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333633 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-oauth-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.333753 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333698 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.333753 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333716 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.333753 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333751 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-service-ca\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333767 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-oauth-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333820 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-trusted-ca-bundle\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.333864 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqzdh\" (UniqueName: \"kubernetes.io/projected/9640041c-9183-48db-b0c6-dd0cc9c192e2-kube-api-access-wqzdh\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334561 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.334533 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334776 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.334615 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-oauth-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334776 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.334637 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-service-ca\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.334938 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.334909 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9640041c-9183-48db-b0c6-dd0cc9c192e2-trusted-ca-bundle\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.336292 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.336272 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-serving-cert\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.336366 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.336306 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9640041c-9183-48db-b0c6-dd0cc9c192e2-console-oauth-config\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.343755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.343738 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqzdh\" (UniqueName: \"kubernetes.io/projected/9640041c-9183-48db-b0c6-dd0cc9c192e2-kube-api-access-wqzdh\") pod \"console-8cc7cdf69-57snr\" (UID: \"9640041c-9183-48db-b0c6-dd0cc9c192e2\") " pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.423998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.423960 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:01:59.580093 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.580067 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8cc7cdf69-57snr"] Jul 17 19:01:59.581696 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:01:59.581667 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9640041c_9183_48db_b0c6_dd0cc9c192e2.slice/crio-babd3862f37a6e8452ea311c1a8dee8cb764e16cb471b6ff4b952b876ee70722 WatchSource:0}: Error finding container babd3862f37a6e8452ea311c1a8dee8cb764e16cb471b6ff4b952b876ee70722: Status 404 returned error can't find the container with id babd3862f37a6e8452ea311c1a8dee8cb764e16cb471b6ff4b952b876ee70722 Jul 17 19:01:59.663144 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.663114 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cc7cdf69-57snr" event={"ID":"9640041c-9183-48db-b0c6-dd0cc9c192e2","Type":"ContainerStarted","Data":"d473b00b7f1d624cee241737916a3471026741aea4a534484f8e10e8ae3ba904"} Jul 17 19:01:59.663273 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.663174 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8cc7cdf69-57snr" event={"ID":"9640041c-9183-48db-b0c6-dd0cc9c192e2","Type":"ContainerStarted","Data":"babd3862f37a6e8452ea311c1a8dee8cb764e16cb471b6ff4b952b876ee70722"} Jul 17 19:01:59.683582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:01:59.683524 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-8cc7cdf69-57snr" podStartSLOduration=0.683507238 podStartE2EDuration="683.507238ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:01:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:01:59.683037327 +0000 UTC m=+401.334417782" watchObservedRunningTime="2026-07-17 19:01:59.683507238 +0000 UTC m=+401.334887692" Jul 17 19:02:09.424707 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:09.424659 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:02:09.425306 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:09.424734 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:02:09.429616 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:09.429593 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:02:09.709917 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:09.709891 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8cc7cdf69-57snr" Jul 17 19:02:24.514465 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.514435 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp"] Jul 17 19:02:24.517942 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.517925 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.521139 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.521105 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 17 19:02:24.521268 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.521163 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-w799f\"" Jul 17 19:02:24.521268 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.521111 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 17 19:02:24.521422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.521409 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 17 19:02:24.522428 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.522414 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 17 19:02:24.529256 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.529229 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp"] Jul 17 19:02:24.557830 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.557801 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.558009 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.557849 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba491637-6554-41a4-95a5-2189d500167f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.558009 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.557911 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vcp6c\" (UniqueName: \"kubernetes.io/projected/ba491637-6554-41a4-95a5-2189d500167f-kube-api-access-vcp6c\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.659117 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.659084 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.659322 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.659127 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba491637-6554-41a4-95a5-2189d500167f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.659322 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.659154 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vcp6c\" (UniqueName: \"kubernetes.io/projected/ba491637-6554-41a4-95a5-2189d500167f-kube-api-access-vcp6c\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.659322 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:24.659240 2584 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 17 19:02:24.659322 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:24.659317 2584 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert podName:ba491637-6554-41a4-95a5-2189d500167f nodeName:}" failed. No retries permitted until 2026-07-17 19:02:25.159296457 +0000 UTC m=+426.810676897 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-s7wcp" (UID: "ba491637-6554-41a4-95a5-2189d500167f") : secret "plugin-serving-cert" not found Jul 17 19:02:24.659835 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.659819 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ba491637-6554-41a4-95a5-2189d500167f-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:24.668288 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:24.668266 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vcp6c\" (UniqueName: \"kubernetes.io/projected/ba491637-6554-41a4-95a5-2189d500167f-kube-api-access-vcp6c\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:25.163863 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:25.163825 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:25.166349 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:25.166330 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/ba491637-6554-41a4-95a5-2189d500167f-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-s7wcp\" (UID: \"ba491637-6554-41a4-95a5-2189d500167f\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:25.428591 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:25.428504 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" Jul 17 19:02:25.562060 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:25.562033 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp"] Jul 17 19:02:25.562588 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:02:25.562560 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba491637_6554_41a4_95a5_2189d500167f.slice/crio-5ca2e596c469162942e4cdb5fdfe7112db18120567c9c7fdffff20c54564861d WatchSource:0}: Error finding container 5ca2e596c469162942e4cdb5fdfe7112db18120567c9c7fdffff20c54564861d: Status 404 returned error can't find the container with id 5ca2e596c469162942e4cdb5fdfe7112db18120567c9c7fdffff20c54564861d Jul 17 19:02:25.782977 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:25.782946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" event={"ID":"ba491637-6554-41a4-95a5-2189d500167f","Type":"ContainerStarted","Data":"5ca2e596c469162942e4cdb5fdfe7112db18120567c9c7fdffff20c54564861d"} Jul 17 19:02:34.791615 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.791576 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6"] Jul 17 19:02:34.801426 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.801395 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.805097 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.805066 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b2sv6\"" Jul 17 19:02:34.813538 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.813507 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6"] Jul 17 19:02:34.859501 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.859465 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.859698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.859515 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8pf5f\" (UniqueName: \"kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.960419 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.960375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8pf5f\" (UniqueName: \"kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.960599 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.960542 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.960999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.960976 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:34.973342 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:34.973312 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8pf5f\" (UniqueName: \"kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:35.116446 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:35.116360 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:35.567033 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:35.566993 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6"] Jul 17 19:02:35.572838 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:35.572813 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6"] Jul 17 19:02:47.904193 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:47.904164 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" event={"ID":"ba491637-6554-41a4-95a5-2189d500167f","Type":"ContainerStarted","Data":"5a01bf76ac48ddc7c19b2d7f3132a0628e5a447f4461d90e3e55e68e793720d0"} Jul 17 19:02:47.922279 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:47.922193 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-s7wcp" podStartSLOduration=1.6564879019999998 podStartE2EDuration="23.922171593s" totalImagesPullingTime="22.265683691s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:02:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:02:47.920991677 +0000 UTC m=+449.572372147" watchObservedRunningTime="2026-07-17 19:02:47.922171593 +0000 UTC m=+449.573552049" Jul 17 19:02:50.295445 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:50.295399 2584 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-4kjh6_kuadrant-system_56b7dd1f-9aea-478a-8917-fa28ff6408c1_0(f6585d84c95c58f69b13765d47519923c3c2c3f2544d00b8ea3388b834edd46d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-4kjh6 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 17 19:02:50.295765 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:50.295471 2584 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-6fb7879bc9-4kjh6_kuadrant-system_56b7dd1f-9aea-478a-8917-fa28ff6408c1_0(f6585d84c95c58f69b13765d47519923c3c2c3f2544d00b8ea3388b834edd46d): error adding pod kuadrant-system_kuadrant-operator-controller-manager-6fb7879bc9-4kjh6 to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:50.917347 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:50.917320 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:50.920037 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:50.920003 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" is forbidden: User \"system:node:ip-10-0-133-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-230.ec2.internal' and this object" podUID="56b7dd1f-9aea-478a-8917-fa28ff6408c1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:50.925954 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:50.925933 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:50.928565 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:50.928544 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" is forbidden: User \"system:node:ip-10-0-133-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-230.ec2.internal' and this object" podUID="56b7dd1f-9aea-478a-8917-fa28ff6408c1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:51.016667 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.016616 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume\") pod \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " Jul 17 19:02:51.016851 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.016688 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8pf5f\" (UniqueName: \"kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f\") pod \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\" (UID: \"56b7dd1f-9aea-478a-8917-fa28ff6408c1\") " Jul 17 19:02:51.016915 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.016849 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "56b7dd1f-9aea-478a-8917-fa28ff6408c1" (UID: "56b7dd1f-9aea-478a-8917-fa28ff6408c1"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:02:51.017069 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.017046 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/56b7dd1f-9aea-478a-8917-fa28ff6408c1-extensions-socket-volume\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:02:51.019241 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.019220 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f" (OuterVolumeSpecName: "kube-api-access-8pf5f") pod "56b7dd1f-9aea-478a-8917-fa28ff6408c1" (UID: "56b7dd1f-9aea-478a-8917-fa28ff6408c1"). InnerVolumeSpecName "kube-api-access-8pf5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:02:51.118152 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.118121 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8pf5f\" (UniqueName: \"kubernetes.io/projected/56b7dd1f-9aea-478a-8917-fa28ff6408c1-kube-api-access-8pf5f\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:02:51.922110 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:51.922078 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:51.925303 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:51.925269 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" is forbidden: User \"system:node:ip-10-0-133-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-230.ec2.internal' and this object" podUID="56b7dd1f-9aea-478a-8917-fa28ff6408c1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:51.930169 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:02:51.930142 2584 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-4kjh6\" is forbidden: User \"system:node:ip-10-0-133-230.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-133-230.ec2.internal' and this object" podUID="56b7dd1f-9aea-478a-8917-fa28ff6408c1" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-4kjh6" Jul 17 19:02:52.904964 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:52.904923 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56b7dd1f-9aea-478a-8917-fa28ff6408c1" path="/var/lib/kubelet/pods/56b7dd1f-9aea-478a-8917-fa28ff6408c1/volumes" Jul 17 19:02:53.582030 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.581988 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:02:53.585837 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.585813 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.588876 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.588857 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b2sv6\"" Jul 17 19:02:53.599144 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.599117 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:02:53.741179 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.741140 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5lbnb\" (UniqueName: \"kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.741179 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.741180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.842463 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.842371 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5lbnb\" (UniqueName: \"kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.842463 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.842411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.842860 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.842842 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.855108 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.855074 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5lbnb\" (UniqueName: \"kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb\") pod \"kuadrant-operator-controller-manager-86878c769f-nmdkv\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:53.896027 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:53.895985 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:54.030747 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:54.030724 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:02:54.033529 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:02:54.033499 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9021f415_4100_4eab_86e0_e068993fbbfd.slice/crio-cea0030ca0f1d1ca84c554f3cacb156f6dd5d0c031bfe75d58de69de888b9e2c WatchSource:0}: Error finding container cea0030ca0f1d1ca84c554f3cacb156f6dd5d0c031bfe75d58de69de888b9e2c: Status 404 returned error can't find the container with id cea0030ca0f1d1ca84c554f3cacb156f6dd5d0c031bfe75d58de69de888b9e2c Jul 17 19:02:54.938411 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:54.938373 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" event={"ID":"9021f415-4100-4eab-86e0-e068993fbbfd","Type":"ContainerStarted","Data":"cea0030ca0f1d1ca84c554f3cacb156f6dd5d0c031bfe75d58de69de888b9e2c"} Jul 17 19:02:58.964368 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:58.964335 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" event={"ID":"9021f415-4100-4eab-86e0-e068993fbbfd","Type":"ContainerStarted","Data":"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965"} Jul 17 19:02:58.964827 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:58.964520 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:02:58.987298 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:02:58.987243 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" podStartSLOduration=1.204258365 podStartE2EDuration="5.987228351s" totalImagesPullingTime="4.782969986s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:02:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:02:58.984782444 +0000 UTC m=+460.636162918" watchObservedRunningTime="2026-07-17 19:02:58.987228351 +0000 UTC m=+460.638608806" Jul 17 19:03:09.971793 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:09.971758 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:03:52.169820 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.169783 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:03:52.172482 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.172467 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.175400 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.175372 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 19:03:52.175565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.175536 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 19:03:52.176861 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.176846 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wnxwq\"" Jul 17 19:03:52.184113 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.184089 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:03:52.294751 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.294716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.294952 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.294783 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hqnw\" (UniqueName: \"kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.395556 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.395514 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9hqnw\" (UniqueName: \"kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.395768 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.395633 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.398136 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.398116 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.407208 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.407188 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hqnw\" (UniqueName: \"kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw\") pod \"maas-controller-6f99c964f4-7nvkz\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.483193 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.483162 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:52.627997 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:52.627970 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:03:52.629330 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:03:52.629303 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod71727794_cd5b_4920_a46b_a8d278f9f429.slice/crio-f82b09f9c9ffa5d853e217ec007fda2b2dd1877a461752d5a57cd2b1df34bde0 WatchSource:0}: Error finding container f82b09f9c9ffa5d853e217ec007fda2b2dd1877a461752d5a57cd2b1df34bde0: Status 404 returned error can't find the container with id f82b09f9c9ffa5d853e217ec007fda2b2dd1877a461752d5a57cd2b1df34bde0 Jul 17 19:03:53.213565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:53.213535 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" event={"ID":"71727794-cd5b-4920-a46b-a8d278f9f429","Type":"ContainerStarted","Data":"f82b09f9c9ffa5d853e217ec007fda2b2dd1877a461752d5a57cd2b1df34bde0"} Jul 17 19:03:55.226078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:55.226038 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" event={"ID":"71727794-cd5b-4920-a46b-a8d278f9f429","Type":"ContainerStarted","Data":"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24"} Jul 17 19:03:55.226483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:55.226155 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:03:55.245631 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:03:55.245583 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" podStartSLOduration=0.990160368 podStartE2EDuration="3.245568504s" totalImagesPullingTime="2.255408136s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:03:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:03:55.244740777 +0000 UTC m=+516.896121244" watchObservedRunningTime="2026-07-17 19:03:55.245568504 +0000 UTC m=+516.896948957" Jul 17 19:04:06.234078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:06.234048 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:04:28.011841 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.011764 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xxtpk"] Jul 17 19:04:28.014130 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.014114 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.016808 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.016784 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 17 19:04:28.016913 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.016827 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-n56hd\"" Jul 17 19:04:28.025010 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.024986 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xxtpk"] Jul 17 19:04:28.025277 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.025251 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1138eed-e019-44fa-a13d-15e5c812fc46-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.025390 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.025369 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxhrx\" (UniqueName: \"kubernetes.io/projected/d1138eed-e019-44fa-a13d-15e5c812fc46-kube-api-access-mxhrx\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.126777 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.126734 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1138eed-e019-44fa-a13d-15e5c812fc46-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.126960 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.126861 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxhrx\" (UniqueName: \"kubernetes.io/projected/d1138eed-e019-44fa-a13d-15e5c812fc46-kube-api-access-mxhrx\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.127479 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.127454 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d1138eed-e019-44fa-a13d-15e5c812fc46-plugins-config-volume\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.136699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.136675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxhrx\" (UniqueName: \"kubernetes.io/projected/d1138eed-e019-44fa-a13d-15e5c812fc46-kube-api-access-mxhrx\") pod \"payload-pre-processing-7677b774bb-xxtpk\" (UID: \"d1138eed-e019-44fa-a13d-15e5c812fc46\") " pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.325448 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.325341 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:28.481885 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:28.481862 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7677b774bb-xxtpk"] Jul 17 19:04:28.484395 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:04:28.484367 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd1138eed_e019_44fa_a13d_15e5c812fc46.slice/crio-0374079d27b480781e9e2b36fee3935c6af9773d2ebd689404cd8ed6228accd6 WatchSource:0}: Error finding container 0374079d27b480781e9e2b36fee3935c6af9773d2ebd689404cd8ed6228accd6: Status 404 returned error can't find the container with id 0374079d27b480781e9e2b36fee3935c6af9773d2ebd689404cd8ed6228accd6 Jul 17 19:04:29.389736 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:29.389697 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" event={"ID":"d1138eed-e019-44fa-a13d-15e5c812fc46","Type":"ContainerStarted","Data":"0374079d27b480781e9e2b36fee3935c6af9773d2ebd689404cd8ed6228accd6"} Jul 17 19:04:31.402444 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:31.402410 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" event={"ID":"d1138eed-e019-44fa-a13d-15e5c812fc46","Type":"ContainerStarted","Data":"8e4f6fcfd812b7c2cae567f09aa24ed184e79bfdbdbddf67ea62d60adbd55875"} Jul 17 19:04:31.402944 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:31.402481 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:04:31.420298 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:31.420255 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" podStartSLOduration=2.230261824 podStartE2EDuration="4.420242427s" totalImagesPullingTime="2.189980603s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:04:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:04:31.419917093 +0000 UTC m=+553.071297548" watchObservedRunningTime="2026-07-17 19:04:31.420242427 +0000 UTC m=+553.071622882" Jul 17 19:04:42.409520 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:42.409470 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" podUID="d1138eed-e019-44fa-a13d-15e5c812fc46" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.134.0.33:9004: connect: connection refused" Jul 17 19:04:52.408997 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:04:52.408963 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7677b774bb-xxtpk" Jul 17 19:05:05.611000 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.610963 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d"] Jul 17 19:05:05.616827 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.616803 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.620030 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.620006 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-4fjw6\"" Jul 17 19:05:05.621519 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.621497 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 17 19:05:05.621669 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.621503 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 17 19:05:05.621669 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.621506 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 17 19:05:05.629609 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.629581 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d"] Jul 17 19:05:05.694925 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.694892 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.695083 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.694943 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgpw\" (UniqueName: \"kubernetes.io/projected/ac6abfad-f4e4-4e9d-9930-545717f67ae9-kube-api-access-wsgpw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.695083 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.694972 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.695165 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.695092 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.695165 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.695121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.695165 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.695144 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796460 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgpw\" (UniqueName: \"kubernetes.io/projected/ac6abfad-f4e4-4e9d-9930-545717f67ae9-kube-api-access-wsgpw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796521 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796603 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796691 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.796978 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.796762 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.797037 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.797014 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.797096 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.797064 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.797151 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.797102 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.799062 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.799041 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ac6abfad-f4e4-4e9d-9930-545717f67ae9-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.799322 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.799303 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ac6abfad-f4e4-4e9d-9930-545717f67ae9-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.806132 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.806110 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgpw\" (UniqueName: \"kubernetes.io/projected/ac6abfad-f4e4-4e9d-9930-545717f67ae9-kube-api-access-wsgpw\") pod \"e2e-trlp-test-simulated-kserve-598466548d-75b4d\" (UID: \"ac6abfad-f4e4-4e9d-9930-545717f67ae9\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:05.928521 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:05.928427 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:06.067860 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:06.067831 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d"] Jul 17 19:05:06.070253 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:05:06.070223 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac6abfad_f4e4_4e9d_9930_545717f67ae9.slice/crio-bd21786840fdd9366ea409f1ce53805084117dcaeeb1fc29161cd06283c9d1c3 WatchSource:0}: Error finding container bd21786840fdd9366ea409f1ce53805084117dcaeeb1fc29161cd06283c9d1c3: Status 404 returned error can't find the container with id bd21786840fdd9366ea409f1ce53805084117dcaeeb1fc29161cd06283c9d1c3 Jul 17 19:05:06.572975 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:06.572936 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" event={"ID":"ac6abfad-f4e4-4e9d-9930-545717f67ae9","Type":"ContainerStarted","Data":"bd21786840fdd9366ea409f1ce53805084117dcaeeb1fc29161cd06283c9d1c3"} Jul 17 19:05:07.579664 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:07.579621 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" event={"ID":"ac6abfad-f4e4-4e9d-9930-545717f67ae9","Type":"ContainerStarted","Data":"d6f8a69c6e57b97217dd9f000ca70137fd8a82e91fe3e38410feb7ff166ec467"} Jul 17 19:05:07.602794 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:07.602739 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" podStartSLOduration=1.488648537 podStartE2EDuration="2.602724024s" totalImagesPullingTime="1.114075487s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:05:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:05:07.601344515 +0000 UTC m=+589.252724973" watchObservedRunningTime="2026-07-17 19:05:07.602724024 +0000 UTC m=+589.254104480" Jul 17 19:05:15.929542 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:15.929498 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:15.930066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:15.929595 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:15.941944 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:15.941922 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:16.633481 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:16.633446 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-75b4d" Jul 17 19:05:23.501903 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.501869 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw"] Jul 17 19:05:23.505076 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.505055 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.507949 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.507927 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jul 17 19:05:23.515665 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.515619 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw"] Jul 17 19:05:23.671867 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.671823 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.672050 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.671881 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.672050 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.671924 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vvhv\" (UniqueName: \"kubernetes.io/projected/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-kube-api-access-6vvhv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.672050 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.671985 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.672050 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.672019 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.672225 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.672096 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773213 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773132 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773213 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773178 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773235 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vvhv\" (UniqueName: \"kubernetes.io/projected/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-kube-api-access-6vvhv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773296 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773328 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773689 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773631 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-home\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773764 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773745 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.773801 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.773774 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-model-cache\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.775504 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.775487 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-dshm\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.775939 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.775921 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.781548 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.781524 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vvhv\" (UniqueName: \"kubernetes.io/projected/9e27b151-f35b-4732-8ac8-fcc91e93ddc0-kube-api-access-6vvhv\") pod \"premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw\" (UID: \"9e27b151-f35b-4732-8ac8-fcc91e93ddc0\") " pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.815545 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.815520 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:23.954236 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:23.954208 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw"] Jul 17 19:05:23.956197 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:05:23.956167 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9e27b151_f35b_4732_8ac8_fcc91e93ddc0.slice/crio-3b4cadadaec7d8a9b51d80eabefd9e446450a93f32bbe6b496d36742a95d3551 WatchSource:0}: Error finding container 3b4cadadaec7d8a9b51d80eabefd9e446450a93f32bbe6b496d36742a95d3551: Status 404 returned error can't find the container with id 3b4cadadaec7d8a9b51d80eabefd9e446450a93f32bbe6b496d36742a95d3551 Jul 17 19:05:24.663053 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:24.663010 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" event={"ID":"9e27b151-f35b-4732-8ac8-fcc91e93ddc0","Type":"ContainerStarted","Data":"55bfe46ab953d386ef35dffcf18ea178e6cd14d9b15fba33fa621d2ba594feae"} Jul 17 19:05:24.663053 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:24.663052 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" event={"ID":"9e27b151-f35b-4732-8ac8-fcc91e93ddc0","Type":"ContainerStarted","Data":"3b4cadadaec7d8a9b51d80eabefd9e446450a93f32bbe6b496d36742a95d3551"} Jul 17 19:05:24.681483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:24.681427 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" podStartSLOduration=1.463973001 podStartE2EDuration="1.681408827s" totalImagesPullingTime="217.435826ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:05:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:05:24.680896229 +0000 UTC m=+606.332276697" watchObservedRunningTime="2026-07-17 19:05:24.681408827 +0000 UTC m=+606.332789282" Jul 17 19:05:33.816194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:33.816157 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:33.816589 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:33.816221 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:33.828612 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:33.828583 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:05:34.722465 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:05:34.722437 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw" Jul 17 19:07:18.470635 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.470539 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:07:18.471125 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.470829 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" podUID="71727794-cd5b-4920-a46b-a8d278f9f429" containerName="manager" containerID="cri-o://369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24" gracePeriod=30 Jul 17 19:07:18.721353 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.721300 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:07:18.864635 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.864601 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert\") pod \"71727794-cd5b-4920-a46b-a8d278f9f429\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " Jul 17 19:07:18.864841 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.864638 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hqnw\" (UniqueName: \"kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw\") pod \"71727794-cd5b-4920-a46b-a8d278f9f429\" (UID: \"71727794-cd5b-4920-a46b-a8d278f9f429\") " Jul 17 19:07:18.866931 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.866902 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "71727794-cd5b-4920-a46b-a8d278f9f429" (UID: "71727794-cd5b-4920-a46b-a8d278f9f429"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:07:18.867030 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.867007 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw" (OuterVolumeSpecName: "kube-api-access-9hqnw") pod "71727794-cd5b-4920-a46b-a8d278f9f429" (UID: "71727794-cd5b-4920-a46b-a8d278f9f429"). InnerVolumeSpecName "kube-api-access-9hqnw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:07:18.965436 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.965408 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/71727794-cd5b-4920-a46b-a8d278f9f429-webhook-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:07:18.965436 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:18.965436 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hqnw\" (UniqueName: \"kubernetes.io/projected/71727794-cd5b-4920-a46b-a8d278f9f429-kube-api-access-9hqnw\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:07:19.219192 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.219151 2584 generic.go:356] "Generic (PLEG): container finished" podID="71727794-cd5b-4920-a46b-a8d278f9f429" containerID="369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24" exitCode=0 Jul 17 19:07:19.219192 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.219194 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" Jul 17 19:07:19.219382 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.219232 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" event={"ID":"71727794-cd5b-4920-a46b-a8d278f9f429","Type":"ContainerDied","Data":"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24"} Jul 17 19:07:19.219382 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.219287 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-7nvkz" event={"ID":"71727794-cd5b-4920-a46b-a8d278f9f429","Type":"ContainerDied","Data":"f82b09f9c9ffa5d853e217ec007fda2b2dd1877a461752d5a57cd2b1df34bde0"} Jul 17 19:07:19.219382 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.219309 2584 scope.go:117] "RemoveContainer" containerID="369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24" Jul 17 19:07:19.228006 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.227986 2584 scope.go:117] "RemoveContainer" containerID="369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24" Jul 17 19:07:19.228316 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:07:19.228297 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24\": container with ID starting with 369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24 not found: ID does not exist" containerID="369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24" Jul 17 19:07:19.228390 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.228323 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24"} err="failed to get container status \"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24\": rpc error: code = NotFound desc = could not find container \"369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24\": container with ID starting with 369efab1c076fc4987577120fa19f571a6afd3781c0279d015a450945366bb24 not found: ID does not exist" Jul 17 19:07:19.247690 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.247668 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:07:19.251736 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:19.251709 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-7nvkz"] Jul 17 19:07:20.906416 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:07:20.906384 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="71727794-cd5b-4920-a46b-a8d278f9f429" path="/var/lib/kubelet/pods/71727794-cd5b-4920-a46b-a8d278f9f429/volumes" Jul 17 19:08:20.051535 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.051502 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:20.051951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.051932 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="71727794-cd5b-4920-a46b-a8d278f9f429" containerName="manager" Jul 17 19:08:20.051951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.051943 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="71727794-cd5b-4920-a46b-a8d278f9f429" containerName="manager" Jul 17 19:08:20.052025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.052006 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="71727794-cd5b-4920-a46b-a8d278f9f429" containerName="manager" Jul 17 19:08:20.055453 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.055438 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.059638 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.059617 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 19:08:20.059817 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.059752 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wnxwq\"" Jul 17 19:08:20.059817 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.059771 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 19:08:20.062494 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.062469 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:20.145219 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.145187 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.145380 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.145321 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk7b9\" (UniqueName: \"kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.245871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.245833 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.246236 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.245944 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk7b9\" (UniqueName: \"kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.248434 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.248413 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.255154 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.255121 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk7b9\" (UniqueName: \"kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9\") pod \"maas-controller-6f99c964f4-rn45g\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.367103 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.367014 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:20.503056 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.503028 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:20.504954 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:08:20.504927 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a065459_f51d_4e7d_bbe6_ce4756f2bc4d.slice/crio-290ee5335c52c0f37d907a24693532b48272c9d52ce587cb4744b08041cc5665 WatchSource:0}: Error finding container 290ee5335c52c0f37d907a24693532b48272c9d52ce587cb4744b08041cc5665: Status 404 returned error can't find the container with id 290ee5335c52c0f37d907a24693532b48272c9d52ce587cb4744b08041cc5665 Jul 17 19:08:20.506326 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.506306 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 19:08:20.527158 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:20.527127 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-rn45g" event={"ID":"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d","Type":"ContainerStarted","Data":"290ee5335c52c0f37d907a24693532b48272c9d52ce587cb4744b08041cc5665"} Jul 17 19:08:21.533982 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:21.533946 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-rn45g" event={"ID":"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d","Type":"ContainerStarted","Data":"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b"} Jul 17 19:08:21.534452 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:21.534062 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:21.552500 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:21.552456 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f99c964f4-rn45g" podStartSLOduration=1.124172964 podStartE2EDuration="1.552442801s" totalImagesPullingTime="428.269837ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:08:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:08:21.550410776 +0000 UTC m=+783.201791225" watchObservedRunningTime="2026-07-17 19:08:21.552442801 +0000 UTC m=+783.203823256" Jul 17 19:08:32.541216 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:32.541188 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:37.163513 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.163478 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:37.163985 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.163715 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-6f99c964f4-rn45g" podUID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" containerName="manager" containerID="cri-o://2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b" gracePeriod=30 Jul 17 19:08:37.418380 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.418312 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:37.515853 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.515821 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert\") pod \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " Jul 17 19:08:37.516120 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.515866 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk7b9\" (UniqueName: \"kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9\") pod \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\" (UID: \"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d\") " Jul 17 19:08:37.518250 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.518220 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" (UID: "4a065459-f51d-4e7d-bbe6-ce4756f2bc4d"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:08:37.518250 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.518230 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9" (OuterVolumeSpecName: "kube-api-access-zk7b9") pod "4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" (UID: "4a065459-f51d-4e7d-bbe6-ce4756f2bc4d"). InnerVolumeSpecName "kube-api-access-zk7b9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:08:37.610894 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.610856 2584 generic.go:356] "Generic (PLEG): container finished" podID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" containerID="2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b" exitCode=0 Jul 17 19:08:37.610894 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.610898 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-rn45g" Jul 17 19:08:37.611175 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.610913 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-rn45g" event={"ID":"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d","Type":"ContainerDied","Data":"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b"} Jul 17 19:08:37.611175 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.610968 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-rn45g" event={"ID":"4a065459-f51d-4e7d-bbe6-ce4756f2bc4d","Type":"ContainerDied","Data":"290ee5335c52c0f37d907a24693532b48272c9d52ce587cb4744b08041cc5665"} Jul 17 19:08:37.611175 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.610996 2584 scope.go:117] "RemoveContainer" containerID="2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b" Jul 17 19:08:37.616980 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.616955 2584 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-webhook-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:08:37.617095 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.616985 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk7b9\" (UniqueName: \"kubernetes.io/projected/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d-kube-api-access-zk7b9\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:08:37.621226 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.621205 2584 scope.go:117] "RemoveContainer" containerID="2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b" Jul 17 19:08:37.621501 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:08:37.621480 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b\": container with ID starting with 2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b not found: ID does not exist" containerID="2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b" Jul 17 19:08:37.621572 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.621509 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b"} err="failed to get container status \"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b\": rpc error: code = NotFound desc = could not find container \"2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b\": container with ID starting with 2542bc91a3cb5f36754e633cbf7ad8f4866d205de27ae69c20a2480b0222f01b not found: ID does not exist" Jul 17 19:08:37.633477 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.633452 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:37.639512 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:37.639485 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-rn45g"] Jul 17 19:08:38.414208 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.414176 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6f99c964f4-qqnt2"] Jul 17 19:08:38.414744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.414548 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" containerName="manager" Jul 17 19:08:38.414744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.414559 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" containerName="manager" Jul 17 19:08:38.414744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.414632 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" containerName="manager" Jul 17 19:08:38.419533 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.419507 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.422323 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.422298 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 17 19:08:38.423547 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.423528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-wnxwq\"" Jul 17 19:08:38.423613 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.423528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 17 19:08:38.426726 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.426703 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-qqnt2"] Jul 17 19:08:38.526579 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.526546 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9dht\" (UniqueName: \"kubernetes.io/projected/cfd230f0-2476-4249-8c2a-2872725fa06a-kube-api-access-n9dht\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.526788 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.526609 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd230f0-2476-4249-8c2a-2872725fa06a-webhook-cert\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.627948 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.627913 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n9dht\" (UniqueName: \"kubernetes.io/projected/cfd230f0-2476-4249-8c2a-2872725fa06a-kube-api-access-n9dht\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.628128 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.628003 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd230f0-2476-4249-8c2a-2872725fa06a-webhook-cert\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.630741 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.630711 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfd230f0-2476-4249-8c2a-2872725fa06a-webhook-cert\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.636724 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.636700 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9dht\" (UniqueName: \"kubernetes.io/projected/cfd230f0-2476-4249-8c2a-2872725fa06a-kube-api-access-n9dht\") pod \"maas-controller-6f99c964f4-qqnt2\" (UID: \"cfd230f0-2476-4249-8c2a-2872725fa06a\") " pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.730838 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.730802 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:38.870766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.870735 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6f99c964f4-qqnt2"] Jul 17 19:08:38.874539 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:08:38.874512 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfd230f0_2476_4249_8c2a_2872725fa06a.slice/crio-14dba8fbd0357074c202c553f0351a1bf33e1cc2b58daded31a9ca3fdd89885a WatchSource:0}: Error finding container 14dba8fbd0357074c202c553f0351a1bf33e1cc2b58daded31a9ca3fdd89885a: Status 404 returned error can't find the container with id 14dba8fbd0357074c202c553f0351a1bf33e1cc2b58daded31a9ca3fdd89885a Jul 17 19:08:38.905006 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:38.904973 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a065459-f51d-4e7d-bbe6-ce4756f2bc4d" path="/var/lib/kubelet/pods/4a065459-f51d-4e7d-bbe6-ce4756f2bc4d/volumes" Jul 17 19:08:39.623779 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:39.623689 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" event={"ID":"cfd230f0-2476-4249-8c2a-2872725fa06a","Type":"ContainerStarted","Data":"46309d1cc28f8cf0fefe91131ae04063a19afd48d42e0d020c9e0374bd49746a"} Jul 17 19:08:39.624142 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:39.623779 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" event={"ID":"cfd230f0-2476-4249-8c2a-2872725fa06a","Type":"ContainerStarted","Data":"14dba8fbd0357074c202c553f0351a1bf33e1cc2b58daded31a9ca3fdd89885a"} Jul 17 19:08:39.624142 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:39.623883 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:08:39.645330 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:39.645280 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" podStartSLOduration=1.174800823 podStartE2EDuration="1.645265627s" totalImagesPullingTime="470.464804ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:08:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:08:39.644045064 +0000 UTC m=+801.295425522" watchObservedRunningTime="2026-07-17 19:08:39.645265627 +0000 UTC m=+801.296646082" Jul 17 19:08:50.631083 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:08:50.631052 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-6f99c964f4-qqnt2" Jul 17 19:13:26.646717 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:26.646631 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:13:26.647256 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:26.646828 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" podUID="9021f415-4100-4eab-86e0-e068993fbbfd" containerName="manager" containerID="cri-o://cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965" gracePeriod=10 Jul 17 19:13:28.807158 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:28.807132 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:13:28.902443 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:28.902351 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5lbnb\" (UniqueName: \"kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb\") pod \"9021f415-4100-4eab-86e0-e068993fbbfd\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " Jul 17 19:13:28.902443 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:28.902405 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume\") pod \"9021f415-4100-4eab-86e0-e068993fbbfd\" (UID: \"9021f415-4100-4eab-86e0-e068993fbbfd\") " Jul 17 19:13:28.902782 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:28.902760 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9021f415-4100-4eab-86e0-e068993fbbfd" (UID: "9021f415-4100-4eab-86e0-e068993fbbfd"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:13:28.904695 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:28.904668 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb" (OuterVolumeSpecName: "kube-api-access-5lbnb") pod "9021f415-4100-4eab-86e0-e068993fbbfd" (UID: "9021f415-4100-4eab-86e0-e068993fbbfd"). InnerVolumeSpecName "kube-api-access-5lbnb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:13:29.004271 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.004243 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5lbnb\" (UniqueName: \"kubernetes.io/projected/9021f415-4100-4eab-86e0-e068993fbbfd-kube-api-access-5lbnb\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:13:29.004271 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.004268 2584 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9021f415-4100-4eab-86e0-e068993fbbfd-extensions-socket-volume\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:13:29.064302 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.064266 2584 generic.go:356] "Generic (PLEG): container finished" podID="9021f415-4100-4eab-86e0-e068993fbbfd" containerID="cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965" exitCode=0 Jul 17 19:13:29.064510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.064342 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" Jul 17 19:13:29.064510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.064358 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" event={"ID":"9021f415-4100-4eab-86e0-e068993fbbfd","Type":"ContainerDied","Data":"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965"} Jul 17 19:13:29.064510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.064434 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv" event={"ID":"9021f415-4100-4eab-86e0-e068993fbbfd","Type":"ContainerDied","Data":"cea0030ca0f1d1ca84c554f3cacb156f6dd5d0c031bfe75d58de69de888b9e2c"} Jul 17 19:13:29.064510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.064464 2584 scope.go:117] "RemoveContainer" containerID="cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965" Jul 17 19:13:29.075555 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.075535 2584 scope.go:117] "RemoveContainer" containerID="cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965" Jul 17 19:13:29.075844 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:13:29.075823 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965\": container with ID starting with cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965 not found: ID does not exist" containerID="cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965" Jul 17 19:13:29.075917 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.075852 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965"} err="failed to get container status \"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965\": rpc error: code = NotFound desc = could not find container \"cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965\": container with ID starting with cd0a58ccdf695e900898b32895a508a4c3fc6a620d5384da5b402e842a129965 not found: ID does not exist" Jul 17 19:13:29.083841 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.083819 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:13:29.089370 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:29.089345 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-nmdkv"] Jul 17 19:13:30.904591 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:13:30.904550 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9021f415-4100-4eab-86e0-e068993fbbfd" path="/var/lib/kubelet/pods/9021f415-4100-4eab-86e0-e068993fbbfd/volumes" Jul 17 19:14:27.809579 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.809537 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5"] Jul 17 19:14:27.810158 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.810138 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9021f415-4100-4eab-86e0-e068993fbbfd" containerName="manager" Jul 17 19:14:27.810236 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.810161 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9021f415-4100-4eab-86e0-e068993fbbfd" containerName="manager" Jul 17 19:14:27.810297 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.810277 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9021f415-4100-4eab-86e0-e068993fbbfd" containerName="manager" Jul 17 19:14:27.813688 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.813668 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:27.817118 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.817099 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-b2sv6\"" Jul 17 19:14:27.827194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.827171 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5"] Jul 17 19:14:27.971208 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.971170 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gzhxc\" (UniqueName: \"kubernetes.io/projected/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-kube-api-access-gzhxc\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:27.971387 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:27.971225 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.072771 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.072693 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gzhxc\" (UniqueName: \"kubernetes.io/projected/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-kube-api-access-gzhxc\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.072771 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.072745 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.073121 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.073101 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.090196 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.090166 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gzhxc\" (UniqueName: \"kubernetes.io/projected/b3c22461-3bf6-4fb4-baa1-a790081e1ab8-kube-api-access-gzhxc\") pod \"kuadrant-operator-controller-manager-86878c769f-qsks5\" (UID: \"b3c22461-3bf6-4fb4-baa1-a790081e1ab8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.123682 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.123624 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.260375 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.260352 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5"] Jul 17 19:14:28.262912 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:14:28.262885 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3c22461_3bf6_4fb4_baa1_a790081e1ab8.slice/crio-b0da84baa0820f4edd19980dc23ce334c795dde16697569d008b6af59bde8153 WatchSource:0}: Error finding container b0da84baa0820f4edd19980dc23ce334c795dde16697569d008b6af59bde8153: Status 404 returned error can't find the container with id b0da84baa0820f4edd19980dc23ce334c795dde16697569d008b6af59bde8153 Jul 17 19:14:28.361758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.361669 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" event={"ID":"b3c22461-3bf6-4fb4-baa1-a790081e1ab8","Type":"ContainerStarted","Data":"d42f0cc387609ae78336f8430eeba071cd73899da987a0819c41202dc0f10683"} Jul 17 19:14:28.361758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.361732 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" event={"ID":"b3c22461-3bf6-4fb4-baa1-a790081e1ab8","Type":"ContainerStarted","Data":"b0da84baa0820f4edd19980dc23ce334c795dde16697569d008b6af59bde8153"} Jul 17 19:14:28.361941 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.361766 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:14:28.382586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:28.382536 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" podStartSLOduration=1.382521631 podStartE2EDuration="1.382521631s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:14:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:14:28.381406061 +0000 UTC m=+1150.032786542" watchObservedRunningTime="2026-07-17 19:14:28.382521631 +0000 UTC m=+1150.033902142" Jul 17 19:14:39.368349 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:14:39.368275 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-qsks5" Jul 17 19:20:52.516829 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.516746 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:20:52.520669 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.520627 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.523521 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.523501 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-a4acef77-openshift-default-dockercfg-2xd9m\"" Jul 17 19:20:52.547594 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.547568 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:20:52.618462 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618422 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618671 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618473 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zmg2r\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618671 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618595 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618807 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618678 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618807 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618716 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618807 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618745 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618807 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618797 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618957 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618858 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.618957 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.618905 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719754 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719801 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719819 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719862 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719880 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zmg2r\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719921 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.719969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719951 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720303 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.719983 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720303 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720010 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720303 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720188 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720474 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720318 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720474 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720377 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720588 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720479 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.720673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.720613 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.722276 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.722254 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.722569 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.722552 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.728304 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.728284 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.728747 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.728726 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zmg2r\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r\") pod \"e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.831907 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.831814 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:52.977885 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.977861 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:20:52.979958 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:20:52.979932 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4fb7413_a131_4016_9791_5f8cda7cc1e3.slice/crio-09a93b37663ecb2bbe63e3e76502836ebd667f58cb9c48e4cf2ae7a33c91e3c1 WatchSource:0}: Error finding container 09a93b37663ecb2bbe63e3e76502836ebd667f58cb9c48e4cf2ae7a33c91e3c1: Status 404 returned error can't find the container with id 09a93b37663ecb2bbe63e3e76502836ebd667f58cb9c48e4cf2ae7a33c91e3c1 Jul 17 19:20:52.982088 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.982057 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:20:52.982185 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.982121 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:20:52.982185 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:52.982156 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:20:53.219122 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:53.219086 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerStarted","Data":"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7"} Jul 17 19:20:53.219326 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:53.219138 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerStarted","Data":"09a93b37663ecb2bbe63e3e76502836ebd667f58cb9c48e4cf2ae7a33c91e3c1"} Jul 17 19:20:53.245405 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:53.245347 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podStartSLOduration=1.245328505 podStartE2EDuration="1.245328505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:20:53.243799094 +0000 UTC m=+1534.895179559" watchObservedRunningTime="2026-07-17 19:20:53.245328505 +0000 UTC m=+1534.896708960" Jul 17 19:20:53.832835 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:53.832795 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:54.839350 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:54.839317 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:55.230509 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:55.230477 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:20:55.231594 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:20:55.231567 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:21:10.232105 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.232071 2584 patch_prober.go:28] interesting pod/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 17 19:21:10.232517 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.232137 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 17 19:21:10.306737 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.306699 2584 generic.go:356] "Generic (PLEG): container finished" podID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerID="ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7" exitCode=0 Jul 17 19:21:10.306910 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.306777 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerDied","Data":"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7"} Jul 17 19:21:10.307206 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.307188 2584 scope.go:117] "RemoveContainer" containerID="ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7" Jul 17 19:21:10.308266 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.308236 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:21:10.308338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.308321 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:21:10.308375 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:10.308364 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:21:11.313300 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:11.313266 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerStarted","Data":"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a"} Jul 17 19:21:11.833136 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:11.832688 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:21:11.833136 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:11.832819 2584 patch_prober.go:28] interesting pod/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 17 19:21:11.833136 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:11.832861 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 17 19:21:12.832317 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:12.832277 2584 patch_prober.go:28] interesting pod/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 17 19:21:12.832891 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:12.832862 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 17 19:21:13.832486 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:13.832443 2584 patch_prober.go:28] interesting pod/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 17 19:21:13.833173 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:13.833136 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 17 19:21:14.832475 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:14.832431 2584 patch_prober.go:28] interesting pod/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" start-of-body= Jul 17 19:21:14.832733 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:14.832508 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.39:15021/healthz/ready\": dial tcp 10.134.0.39:15021: connect: connection refused" Jul 17 19:21:15.836658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:15.836610 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:21:15.837124 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:15.837107 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:21:15.837745 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:21:15.837719 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:22:23.991237 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:23.991132 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:22:23.991789 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:23.991392 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" containerID="cri-o://85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a" gracePeriod=30 Jul 17 19:22:29.250057 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.250035 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:22:29.388536 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388502 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.388750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388547 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zmg2r\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.388750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388576 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.388750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388671 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.388750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388721 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.388750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388750 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.389036 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388777 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.389036 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388802 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.389036 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.388841 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket\") pod \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\" (UID: \"e4fb7413-a131-4016-9791-5f8cda7cc1e3\") " Jul 17 19:22:29.389036 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389020 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:22:29.389387 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389316 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:22:29.389387 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389370 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.389536 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389476 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:22:29.389594 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389577 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:22:29.389674 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.389589 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:22:29.391084 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.391058 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r" (OuterVolumeSpecName: "kube-api-access-zmg2r") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "kube-api-access-zmg2r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:22:29.391195 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.391161 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:22:29.391425 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.391404 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:22:29.391531 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.391424 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e4fb7413-a131-4016-9791-5f8cda7cc1e3" (UID: "e4fb7413-a131-4016-9791-5f8cda7cc1e3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:22:29.490218 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490182 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zmg2r\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-kube-api-access-zmg2r\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490218 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490214 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490218 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490224 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490233 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490242 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490249 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e4fb7413-a131-4016-9791-5f8cda7cc1e3-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490256 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.490456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.490263 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e4fb7413-a131-4016-9791-5f8cda7cc1e3-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:22:29.705094 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.705055 2584 generic.go:356] "Generic (PLEG): container finished" podID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerID="85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a" exitCode=0 Jul 17 19:22:29.705255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.705110 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" Jul 17 19:22:29.705255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.705132 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerDied","Data":"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a"} Jul 17 19:22:29.705255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.705188 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww" event={"ID":"e4fb7413-a131-4016-9791-5f8cda7cc1e3","Type":"ContainerDied","Data":"09a93b37663ecb2bbe63e3e76502836ebd667f58cb9c48e4cf2ae7a33c91e3c1"} Jul 17 19:22:29.705255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.705209 2584 scope.go:117] "RemoveContainer" containerID="85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a" Jul 17 19:22:29.715848 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.715826 2584 scope.go:117] "RemoveContainer" containerID="ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7" Jul 17 19:22:29.729807 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.729783 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:22:29.730704 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.730689 2584 scope.go:117] "RemoveContainer" containerID="85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a" Jul 17 19:22:29.730980 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:22:29.730965 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a\": container with ID starting with 85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a not found: ID does not exist" containerID="85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a" Jul 17 19:22:29.731017 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.730988 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a"} err="failed to get container status \"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a\": rpc error: code = NotFound desc = could not find container \"85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a\": container with ID starting with 85b2258890c8c114490c2916a89b229975bd90a5d65974024380f7cd12189a6a not found: ID does not exist" Jul 17 19:22:29.731017 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.731011 2584 scope.go:117] "RemoveContainer" containerID="ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7" Jul 17 19:22:29.731213 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:22:29.731194 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7\": container with ID starting with ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7 not found: ID does not exist" containerID="ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7" Jul 17 19:22:29.731252 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.731220 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7"} err="failed to get container status \"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7\": rpc error: code = NotFound desc = could not find container \"ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7\": container with ID starting with ead99f805934006cb29ddc6e99e2ec7aedc010a061e807cf0b2aea0ed3eaa7a7 not found: ID does not exist" Jul 17 19:22:29.735358 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:29.735336 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-a4acef77-openshift-default-65f4dd8967-xgdww"] Jul 17 19:22:30.904522 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:22:30.904488 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" path="/var/lib/kubelet/pods/e4fb7413-a131-4016-9791-5f8cda7cc1e3/volumes" Jul 17 19:23:36.845126 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845085 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:23:36.845617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845509 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.845617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845522 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.845617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845561 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.845617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845569 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.845887 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845656 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.845887 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.845669 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4fb7413-a131-4016-9791-5f8cda7cc1e3" containerName="istio-proxy" Jul 17 19:23:36.850524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.850497 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.853365 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.853337 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-690b208d-openshift-default-dockercfg-47sds\"" Jul 17 19:23:36.860720 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.860672 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:23:36.951915 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.951871 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952112 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952084 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952191 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952134 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952304 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952289 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952365 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952322 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952365 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952349 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kglz\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952480 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952456 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952528 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952499 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:36.952570 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:36.952528 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054100 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054063 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054110 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054193 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054211 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054230 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7kglz\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054255 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054285 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054272 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054603 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054298 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054603 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054757 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054626 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054757 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054670 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054757 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054733 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.054874 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.054828 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.055071 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.055050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.056696 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.056676 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.056942 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.056922 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.062527 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.062498 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.062793 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.062774 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kglz\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz\") pod \"e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.164279 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.164173 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:37.313575 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.313548 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:23:37.315980 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:23:37.315936 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd17ed17e_88ce_4810_9336_5b3db76c4d0f.slice/crio-01a0e01f3036a2bcaf1e562245ba83aceff534ad607f274343037076e4fc15a6 WatchSource:0}: Error finding container 01a0e01f3036a2bcaf1e562245ba83aceff534ad607f274343037076e4fc15a6: Status 404 returned error can't find the container with id 01a0e01f3036a2bcaf1e562245ba83aceff534ad607f274343037076e4fc15a6 Jul 17 19:23:37.318321 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.318285 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:37.318424 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.318375 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:37.318464 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:37.318419 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:38.037600 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:38.037563 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerStarted","Data":"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4"} Jul 17 19:23:38.038056 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:38.037663 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerStarted","Data":"01a0e01f3036a2bcaf1e562245ba83aceff534ad607f274343037076e4fc15a6"} Jul 17 19:23:38.067543 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:38.067490 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podStartSLOduration=2.067467119 podStartE2EDuration="2.067467119s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:23:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:23:38.065672041 +0000 UTC m=+1699.717052490" watchObservedRunningTime="2026-07-17 19:23:38.067467119 +0000 UTC m=+1699.718847574" Jul 17 19:23:38.165110 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:38.165070 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:38.169999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:38.169971 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:39.042876 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:39.042793 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:39.043837 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:39.043817 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:53.115078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.115038 2584 generic.go:356] "Generic (PLEG): container finished" podID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerID="e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4" exitCode=0 Jul 17 19:23:53.115520 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.115114 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerDied","Data":"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4"} Jul 17 19:23:53.115712 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.115695 2584 scope.go:117] "RemoveContainer" containerID="e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4" Jul 17 19:23:53.116924 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.116897 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:53.116996 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.116980 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:53.117044 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:53.117031 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:23:54.043246 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:54.043199 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:54.125524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:54.125489 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerStarted","Data":"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3"} Jul 17 19:23:54.164811 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:54.164784 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:54.165660 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:54.165619 2584 patch_prober.go:28] interesting pod/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 17 19:23:54.165836 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:54.165693 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 17 19:23:55.164730 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:55.164691 2584 patch_prober.go:28] interesting pod/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 17 19:23:55.165129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:55.164749 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 17 19:23:56.164628 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:56.164589 2584 patch_prober.go:28] interesting pod/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 17 19:23:56.164835 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:56.164677 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 17 19:23:57.165341 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:57.165301 2584 patch_prober.go:28] interesting pod/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" start-of-body= Jul 17 19:23:57.165844 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:57.165362 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.40:15021/healthz/ready\": dial tcp 10.134.0.40:15021: connect: connection refused" Jul 17 19:23:58.169089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:58.169053 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:58.169480 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:58.169319 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:23:58.170018 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:23:58.170002 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:24:59.174234 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:24:59.174195 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:24:59.174810 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:24:59.174421 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" containerID="cri-o://ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3" gracePeriod=30 Jul 17 19:25:04.434907 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.434870 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:25:04.479191 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.479160 2584 generic.go:356] "Generic (PLEG): container finished" podID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerID="ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3" exitCode=0 Jul 17 19:25:04.479378 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.479202 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerDied","Data":"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3"} Jul 17 19:25:04.479378 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.479224 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" Jul 17 19:25:04.479378 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.479250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb" event={"ID":"d17ed17e-88ce-4810-9336-5b3db76c4d0f","Type":"ContainerDied","Data":"01a0e01f3036a2bcaf1e562245ba83aceff534ad607f274343037076e4fc15a6"} Jul 17 19:25:04.479378 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.479279 2584 scope.go:117] "RemoveContainer" containerID="ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3" Jul 17 19:25:04.489308 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.489279 2584 scope.go:117] "RemoveContainer" containerID="e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4" Jul 17 19:25:04.500511 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.500478 2584 scope.go:117] "RemoveContainer" containerID="ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3" Jul 17 19:25:04.501024 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:25:04.500991 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3\": container with ID starting with ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3 not found: ID does not exist" containerID="ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3" Jul 17 19:25:04.501139 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.501037 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3"} err="failed to get container status \"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3\": rpc error: code = NotFound desc = could not find container \"ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3\": container with ID starting with ec5ad86bf4dfb9632ccb8839f7a04746bbd1ccb093d251750431e4c4190fc9e3 not found: ID does not exist" Jul 17 19:25:04.501139 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.501064 2584 scope.go:117] "RemoveContainer" containerID="e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4" Jul 17 19:25:04.501535 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:25:04.501512 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4\": container with ID starting with e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4 not found: ID does not exist" containerID="e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4" Jul 17 19:25:04.501661 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.501536 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4"} err="failed to get container status \"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4\": rpc error: code = NotFound desc = could not find container \"e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4\": container with ID starting with e4a0cb089402c3234c118d71f1fa28a87dab7fb095d6c07d6ad141d38a8852f4 not found: ID does not exist" Jul 17 19:25:04.562566 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562524 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.562787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562579 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.562787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562614 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.562787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562688 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.562787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562711 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.563002 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562889 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.563002 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562958 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.563002 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.562886 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:25:04.563153 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563012 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.563153 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563074 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7kglz\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz\") pod \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\" (UID: \"d17ed17e-88ce-4810-9336-5b3db76c4d0f\") " Jul 17 19:25:04.563153 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563085 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:25:04.563153 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563099 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:25:04.563395 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563371 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:25:04.563625 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563508 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.563709 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563661 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.563709 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563695 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.563819 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563718 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.563819 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.563722 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:25:04.565201 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.565170 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:25:04.565201 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.565187 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:25:04.565694 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.565668 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:25:04.565805 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.565726 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz" (OuterVolumeSpecName: "kube-api-access-7kglz") pod "d17ed17e-88ce-4810-9336-5b3db76c4d0f" (UID: "d17ed17e-88ce-4810-9336-5b3db76c4d0f"). InnerVolumeSpecName "kube-api-access-7kglz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:25:04.664282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.664231 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7kglz\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-kube-api-access-7kglz\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.664282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.664271 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.664282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.664281 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.664282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.664289 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.664282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.664297 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d17ed17e-88ce-4810-9336-5b3db76c4d0f-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:25:04.804118 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.804070 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:25:04.808583 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.808552 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-690b208d-openshift-default-5c5b65957c-h72hb"] Jul 17 19:25:04.904554 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:25:04.904517 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" path="/var/lib/kubelet/pods/d17ed17e-88ce-4810-9336-5b3db76c4d0f/volumes" Jul 17 19:27:58.169538 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.169507 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:27:58.170031 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.169982 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:27:58.170031 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.169996 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:27:58.170031 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.170010 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:27:58.170031 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.170015 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:27:58.170163 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.170104 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:27:58.173253 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.173236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.176055 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.176028 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-6d51d6be-openshift-default-dockercfg-5gcvw\"" Jul 17 19:27:58.199139 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.199107 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:27:58.303556 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303514 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303556 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303561 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303600 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303628 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6ztl\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303760 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303911 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303794 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303911 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303834 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.303911 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303860 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.304012 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.303931 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404606 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404572 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404606 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404613 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404629 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404674 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6ztl\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404749 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404780 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404805 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.404871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404828 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405205 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.404866 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405205 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.405062 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405205 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.405160 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405311 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.405251 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405379 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.405354 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.405507 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.405424 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.407293 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.407273 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.407526 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.407505 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.413148 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.413122 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.413252 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.413226 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6ztl\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl\") pod \"e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.483567 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.483534 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:58.629739 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.629714 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:27:58.631786 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:27:58.631756 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod72e4e60d_4d92_49e9_8adb_06d5b27baffb.slice/crio-604c458d7ac50c38327227395ac9fe9b8281bdb575089c6040766f4f8e851e7c WatchSource:0}: Error finding container 604c458d7ac50c38327227395ac9fe9b8281bdb575089c6040766f4f8e851e7c: Status 404 returned error can't find the container with id 604c458d7ac50c38327227395ac9fe9b8281bdb575089c6040766f4f8e851e7c Jul 17 19:27:58.633958 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.633924 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:27:58.634028 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.633992 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:27:58.634028 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:58.634020 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:27:59.332230 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:59.332184 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerStarted","Data":"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413"} Jul 17 19:27:59.332735 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:59.332250 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerStarted","Data":"604c458d7ac50c38327227395ac9fe9b8281bdb575089c6040766f4f8e851e7c"} Jul 17 19:27:59.355610 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:59.355553 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podStartSLOduration=1.355535756 podStartE2EDuration="1.355535756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:27:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:27:59.35279151 +0000 UTC m=+1961.004171966" watchObservedRunningTime="2026-07-17 19:27:59.355535756 +0000 UTC m=+1961.006916211" Jul 17 19:27:59.484029 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:59.483993 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:27:59.488510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:27:59.488486 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:00.338711 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:00.338676 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:00.339763 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:00.339739 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:14.406899 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.406865 2584 generic.go:356] "Generic (PLEG): container finished" podID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerID="4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413" exitCode=0 Jul 17 19:28:14.407303 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.406922 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerDied","Data":"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413"} Jul 17 19:28:14.407303 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.407282 2584 scope.go:117] "RemoveContainer" containerID="4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413" Jul 17 19:28:14.408273 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.408243 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:14.408375 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.408324 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:14.408449 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:14.408372 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:15.339041 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:15.339006 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:15.414101 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:15.414070 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerStarted","Data":"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb"} Jul 17 19:28:15.483744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:15.483712 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:15.484658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:15.484624 2584 patch_prober.go:28] interesting pod/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 17 19:28:15.484787 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:15.484728 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 17 19:28:16.483948 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:16.483915 2584 patch_prober.go:28] interesting pod/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 17 19:28:16.484355 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:16.483974 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 17 19:28:17.484507 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:17.484469 2584 patch_prober.go:28] interesting pod/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 17 19:28:17.484956 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:17.484525 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 17 19:28:18.484927 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:18.484893 2584 patch_prober.go:28] interesting pod/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 17 19:28:18.485326 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:18.484949 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 17 19:28:19.484778 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:19.484744 2584 patch_prober.go:28] interesting pod/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" start-of-body= Jul 17 19:28:19.484961 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:19.484800 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.41:15021/healthz/ready\": dial tcp 10.134.0.41:15021: connect: connection refused" Jul 17 19:28:20.488474 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:20.488442 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:20.488944 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:20.488680 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:20.489457 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:20.489441 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:38.757565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:38.757525 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:28:38.758201 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:38.757817 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" containerID="cri-o://3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb" gracePeriod=30 Jul 17 19:28:44.011892 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.011869 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:44.053590 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053513 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053590 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053549 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6ztl\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053590 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053576 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053602 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053656 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053730 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053825 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053844 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:28:44.053873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053857 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.054185 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.053887 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data\") pod \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\" (UID: \"72e4e60d-4d92-49e9-8adb-06d5b27baffb\") " Jul 17 19:28:44.054185 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054074 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:28:44.054185 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054160 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:28:44.054345 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054314 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.054408 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054353 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.054408 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054368 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.054670 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054624 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:28:44.054868 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.054848 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data" (OuterVolumeSpecName: "istio-data") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:28:44.056008 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.055986 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:28:44.056322 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.056307 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl" (OuterVolumeSpecName: "kube-api-access-p6ztl") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "kube-api-access-p6ztl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:28:44.056847 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.056828 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token" (OuterVolumeSpecName: "istio-token") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:28:44.056847 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.056829 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "72e4e60d-4d92-49e9-8adb-06d5b27baffb" (UID: "72e4e60d-4d92-49e9-8adb-06d5b27baffb"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:28:44.155371 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155341 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6ztl\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-kube-api-access-p6ztl\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.155371 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155367 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.155371 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155377 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.155591 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155385 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.155591 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155394 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.155591 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.155402 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/72e4e60d-4d92-49e9-8adb-06d5b27baffb-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:28:44.564332 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.564296 2584 generic.go:356] "Generic (PLEG): container finished" podID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerID="3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb" exitCode=0 Jul 17 19:28:44.564582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.564339 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerDied","Data":"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb"} Jul 17 19:28:44.564582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.564356 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" Jul 17 19:28:44.564582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.564406 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9" event={"ID":"72e4e60d-4d92-49e9-8adb-06d5b27baffb","Type":"ContainerDied","Data":"604c458d7ac50c38327227395ac9fe9b8281bdb575089c6040766f4f8e851e7c"} Jul 17 19:28:44.564582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.564428 2584 scope.go:117] "RemoveContainer" containerID="3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb" Jul 17 19:28:44.573778 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.573760 2584 scope.go:117] "RemoveContainer" containerID="4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413" Jul 17 19:28:44.581778 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.581758 2584 scope.go:117] "RemoveContainer" containerID="3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb" Jul 17 19:28:44.582032 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:28:44.582013 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb\": container with ID starting with 3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb not found: ID does not exist" containerID="3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb" Jul 17 19:28:44.582095 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.582041 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb"} err="failed to get container status \"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb\": rpc error: code = NotFound desc = could not find container \"3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb\": container with ID starting with 3a15fc2be243c48cd09681e6b9d8419844caffc8dfbda38dc27a7c6f9e7506fb not found: ID does not exist" Jul 17 19:28:44.582095 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.582058 2584 scope.go:117] "RemoveContainer" containerID="4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413" Jul 17 19:28:44.582302 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:28:44.582283 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413\": container with ID starting with 4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413 not found: ID does not exist" containerID="4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413" Jul 17 19:28:44.582349 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.582308 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413"} err="failed to get container status \"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413\": rpc error: code = NotFound desc = could not find container \"4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413\": container with ID starting with 4f50d89159ca9648a65685c519f144c9bb2f50e3dc1f5e2c924f7c9b33ffd413 not found: ID does not exist" Jul 17 19:28:44.586364 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.586343 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:28:44.590115 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.590097 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-6d51d6be-openshift-default-5d654c8cf5-b5cb9"] Jul 17 19:28:44.905324 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:44.905243 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" path="/var/lib/kubelet/pods/72e4e60d-4d92-49e9-8adb-06d5b27baffb/volumes" Jul 17 19:28:58.835580 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.835547 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:28:58.836034 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.835975 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:28:58.836034 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.835986 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:28:58.836108 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.836055 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:28:58.836108 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.836064 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:28:58.836108 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.836071 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d17ed17e-88ce-4810-9336-5b3db76c4d0f" containerName="istio-proxy" Jul 17 19:28:58.839192 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.839171 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.842198 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.842176 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-45e4d318-openshift-default-dockercfg-kx87c\"" Jul 17 19:28:58.853171 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.853151 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:28:58.998736 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998698 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.998914 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998753 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.998914 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998796 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tqx5f\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.998914 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998826 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.998914 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998893 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.999132 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998947 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.999132 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.998988 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.999132 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.999022 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:58.999305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:58.999217 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100199 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100116 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100199 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100161 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100201 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100218 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tqx5f\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100242 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100294 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100320 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100375 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100417 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100412 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100808 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100540 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100808 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100685 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.100808 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100742 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.101133 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100842 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.101133 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.100953 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.102548 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.102526 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.102924 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.102907 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.108342 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.108320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.108946 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.108926 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tqx5f\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f\") pod \"e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.149055 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.149032 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:28:59.288713 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.288691 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:28:59.290611 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:28:59.290588 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcd190a2a_00d7_49c4_b9a1_a9995edaee1d.slice/crio-1c1f1192d37f4f094bc2be9b85774745964234deb299269d83236f867f09e789 WatchSource:0}: Error finding container 1c1f1192d37f4f094bc2be9b85774745964234deb299269d83236f867f09e789: Status 404 returned error can't find the container with id 1c1f1192d37f4f094bc2be9b85774745964234deb299269d83236f867f09e789 Jul 17 19:28:59.293171 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.293142 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:59.293239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.293204 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:59.293239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.293232 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:28:59.640195 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.640155 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerStarted","Data":"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093"} Jul 17 19:28:59.640383 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.640221 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerStarted","Data":"1c1f1192d37f4f094bc2be9b85774745964234deb299269d83236f867f09e789"} Jul 17 19:28:59.662168 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:28:59.662119 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podStartSLOduration=1.662104325 podStartE2EDuration="1.662104325s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:28:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:28:59.660080255 +0000 UTC m=+2021.311460729" watchObservedRunningTime="2026-07-17 19:28:59.662104325 +0000 UTC m=+2021.313484780" Jul 17 19:29:00.149800 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:00.149767 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:00.154525 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:00.154499 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:00.646733 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:00.646700 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:00.647687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:00.647665 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:14.406977 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.406933 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:29:14.407593 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.407572 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:29:14.407700 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.407598 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="72e4e60d-4d92-49e9-8adb-06d5b27baffb" containerName="istio-proxy" Jul 17 19:29:14.410479 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.410457 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.414712 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.414670 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-45e4d318\"/\"kube-root-ca.crt\"" Jul 17 19:29:14.414712 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.414683 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-45e4d318\"/\"openshift-service-ca.crt\"" Jul 17 19:29:14.414924 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.414685 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-45e4d318\"/\"e2e-lifecycle-model-45e4d318-kserve-self-signed-certs\"" Jul 17 19:29:14.414924 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.414786 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-45e4d318\"/\"default-dockercfg-8tbbz\"" Jul 17 19:29:14.418965 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.418941 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:29:14.454549 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454512 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwlbv\" (UniqueName: \"kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.454549 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454552 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.454824 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454584 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.454824 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454690 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.454824 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454729 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.454824 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.454787 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555499 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555414 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555692 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555571 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hwlbv\" (UniqueName: \"kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555692 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555611 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555826 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555692 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555826 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555739 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.555826 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.555769 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.556275 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.556107 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.556275 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.556117 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.556275 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.556174 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.558106 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.558078 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.558208 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.558167 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.564119 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.564097 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwlbv\" (UniqueName: \"kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv\") pod \"e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.717148 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.717107 2584 generic.go:356] "Generic (PLEG): container finished" podID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerID="905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093" exitCode=0 Jul 17 19:29:14.717379 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.717187 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerDied","Data":"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093"} Jul 17 19:29:14.717702 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.717674 2584 scope.go:117] "RemoveContainer" containerID="905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093" Jul 17 19:29:14.718864 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.718837 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:29:14.718970 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.718915 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:29:14.719037 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.718978 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:29:14.735837 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.735804 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:14.895032 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:14.895001 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:29:14.896722 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:29:14.896693 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod511854a0_0e19_45d2_8945_0090c5eef0e4.slice/crio-079ea85a33c16f995644ab211c5de35700aabeffae1053c68560d0118ef8ed80 WatchSource:0}: Error finding container 079ea85a33c16f995644ab211c5de35700aabeffae1053c68560d0118ef8ed80: Status 404 returned error can't find the container with id 079ea85a33c16f995644ab211c5de35700aabeffae1053c68560d0118ef8ed80 Jul 17 19:29:15.647508 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:15.647465 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:15.728716 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:15.728676 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerStarted","Data":"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58"} Jul 17 19:29:15.731074 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:15.731035 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" event={"ID":"511854a0-0e19-45d2-8945-0090c5eef0e4","Type":"ContainerStarted","Data":"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0"} Jul 17 19:29:15.731248 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:15.731081 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" event={"ID":"511854a0-0e19-45d2-8945-0090c5eef0e4","Type":"ContainerStarted","Data":"079ea85a33c16f995644ab211c5de35700aabeffae1053c68560d0118ef8ed80"} Jul 17 19:29:15.768241 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:15.768179 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" podStartSLOduration=1.768158406 podStartE2EDuration="1.768158406s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:29:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:29:15.767357709 +0000 UTC m=+2037.418738165" watchObservedRunningTime="2026-07-17 19:29:15.768158406 +0000 UTC m=+2037.419538863" Jul 17 19:29:16.149931 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:16.149891 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:16.151157 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:16.151131 2584 patch_prober.go:28] interesting pod/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 17 19:29:16.151217 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:16.151184 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 17 19:29:17.149607 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:17.149573 2584 patch_prober.go:28] interesting pod/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 17 19:29:17.150036 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:17.149637 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 17 19:29:18.149949 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:18.149908 2584 patch_prober.go:28] interesting pod/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 17 19:29:18.150360 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:18.149964 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 17 19:29:19.150267 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:19.150221 2584 patch_prober.go:28] interesting pod/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" start-of-body= Jul 17 19:29:19.150681 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:19.150304 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.42:15021/healthz/ready\": dial tcp 10.134.0.42:15021: connect: connection refused" Jul 17 19:29:20.154265 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:20.154234 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:20.154697 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:20.154524 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:20.155389 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:20.155370 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:29:24.736340 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:24.736291 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:24.736794 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:24.736354 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:24.748944 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:24.748916 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:24.792901 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:24.792869 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:29:54.104266 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:29:54.102612 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:30:09.129194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.129135 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" podUID="511854a0-0e19-45d2-8945-0090c5eef0e4" containerName="main" containerID="cri-o://fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0" gracePeriod=45 Jul 17 19:30:09.384058 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.384000 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:30:09.526547 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526522 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.526770 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526558 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hwlbv\" (UniqueName: \"kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.526770 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526583 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.526770 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526616 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.526770 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526636 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.526770 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526719 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm\") pod \"511854a0-0e19-45d2-8945-0090c5eef0e4\" (UID: \"511854a0-0e19-45d2-8945-0090c5eef0e4\") " Jul 17 19:30:09.527025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526871 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home" (OuterVolumeSpecName: "home") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:09.527025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526889 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:09.527025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.526911 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache" (OuterVolumeSpecName: "model-cache") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:09.527172 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.527095 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-home\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:09.527172 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.527112 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-tmp-dir\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:09.527172 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.527123 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-model-cache\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:09.529005 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.528976 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:30:09.529005 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.528985 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv" (OuterVolumeSpecName: "kube-api-access-hwlbv") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "kube-api-access-hwlbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:30:09.529152 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.529017 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm" (OuterVolumeSpecName: "dshm") pod "511854a0-0e19-45d2-8945-0090c5eef0e4" (UID: "511854a0-0e19-45d2-8945-0090c5eef0e4"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:09.627716 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.627673 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hwlbv\" (UniqueName: \"kubernetes.io/projected/511854a0-0e19-45d2-8945-0090c5eef0e4-kube-api-access-hwlbv\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:09.627716 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.627712 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/511854a0-0e19-45d2-8945-0090c5eef0e4-tls-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:09.627716 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:09.627724 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/511854a0-0e19-45d2-8945-0090c5eef0e4-dshm\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:10.013565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.013524 2584 generic.go:356] "Generic (PLEG): container finished" podID="511854a0-0e19-45d2-8945-0090c5eef0e4" containerID="fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0" exitCode=0 Jul 17 19:30:10.013565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.013564 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" Jul 17 19:30:10.013846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.013614 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" event={"ID":"511854a0-0e19-45d2-8945-0090c5eef0e4","Type":"ContainerDied","Data":"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0"} Jul 17 19:30:10.013846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.013685 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp" event={"ID":"511854a0-0e19-45d2-8945-0090c5eef0e4","Type":"ContainerDied","Data":"079ea85a33c16f995644ab211c5de35700aabeffae1053c68560d0118ef8ed80"} Jul 17 19:30:10.013846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.013707 2584 scope.go:117] "RemoveContainer" containerID="fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0" Jul 17 19:30:10.025804 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.024210 2584 scope.go:117] "RemoveContainer" containerID="fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0" Jul 17 19:30:10.026133 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:30:10.026107 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0\": container with ID starting with fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0 not found: ID does not exist" containerID="fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0" Jul 17 19:30:10.026228 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.026144 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0"} err="failed to get container status \"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0\": rpc error: code = NotFound desc = could not find container \"fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0\": container with ID starting with fc97dfb7a8da52ba74b609b30efc2fe9bba8a7872add9ce438d497b6721337b0 not found: ID does not exist" Jul 17 19:30:10.036828 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.036809 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:30:10.040864 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.040844 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-45e4d318/e2e-lifecycle-model-45e4d318-kserve-7d45bfb67f-9zzcp"] Jul 17 19:30:10.905150 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:10.905117 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="511854a0-0e19-45d2-8945-0090c5eef0e4" path="/var/lib/kubelet/pods/511854a0-0e19-45d2-8945-0090c5eef0e4/volumes" Jul 17 19:30:41.209257 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:41.209216 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:30:41.209897 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:41.209444 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" containerID="cri-o://050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58" gracePeriod=30 Jul 17 19:30:46.462587 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.462551 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:30:46.579789 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579695 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.579789 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579744 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tqx5f\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.579789 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579789 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579816 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579865 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579951 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.579994 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580032 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580064 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580089 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580090 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy\") pod \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\" (UID: \"cd190a2a-00d7-49c4-b9a1-a9995edaee1d\") " Jul 17 19:30:46.580439 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580165 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:46.580439 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580208 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:46.580439 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580426 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:30:46.580589 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580499 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.580589 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580518 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.580589 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580532 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.580775 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.580747 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data" (OuterVolumeSpecName: "istio-data") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:46.582262 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.582236 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:30:46.582586 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.582572 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:30:46.582766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.582747 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token" (OuterVolumeSpecName: "istio-token") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:30:46.582851 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.582761 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f" (OuterVolumeSpecName: "kube-api-access-tqx5f") pod "cd190a2a-00d7-49c4-b9a1-a9995edaee1d" (UID: "cd190a2a-00d7-49c4-b9a1-a9995edaee1d"). InnerVolumeSpecName "kube-api-access-tqx5f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:30:46.681162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681113 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.681162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681149 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.681162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681164 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.681414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681175 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.681414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681187 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tqx5f\" (UniqueName: \"kubernetes.io/projected/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-kube-api-access-tqx5f\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:46.681414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:46.681200 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/cd190a2a-00d7-49c4-b9a1-a9995edaee1d-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:30:47.200540 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.200514 2584 generic.go:356] "Generic (PLEG): container finished" podID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerID="050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58" exitCode=0 Jul 17 19:30:47.200711 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.200558 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" Jul 17 19:30:47.200711 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.200597 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerDied","Data":"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58"} Jul 17 19:30:47.200711 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.200676 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv" event={"ID":"cd190a2a-00d7-49c4-b9a1-a9995edaee1d","Type":"ContainerDied","Data":"1c1f1192d37f4f094bc2be9b85774745964234deb299269d83236f867f09e789"} Jul 17 19:30:47.200711 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.200703 2584 scope.go:117] "RemoveContainer" containerID="050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58" Jul 17 19:30:47.209676 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.209626 2584 scope.go:117] "RemoveContainer" containerID="905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093" Jul 17 19:30:47.218814 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.218790 2584 scope.go:117] "RemoveContainer" containerID="050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58" Jul 17 19:30:47.219188 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:30:47.219166 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58\": container with ID starting with 050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58 not found: ID does not exist" containerID="050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58" Jul 17 19:30:47.219307 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.219201 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58"} err="failed to get container status \"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58\": rpc error: code = NotFound desc = could not find container \"050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58\": container with ID starting with 050f6d3d48d38c8027ab2a9077b4c8ecc288efbad012747482161a9a9b367e58 not found: ID does not exist" Jul 17 19:30:47.219307 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.219223 2584 scope.go:117] "RemoveContainer" containerID="905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093" Jul 17 19:30:47.219503 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:30:47.219483 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093\": container with ID starting with 905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093 not found: ID does not exist" containerID="905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093" Jul 17 19:30:47.219562 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.219511 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093"} err="failed to get container status \"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093\": rpc error: code = NotFound desc = could not find container \"905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093\": container with ID starting with 905a40b147ccf205d64706331179486c6a000cfec6418f86eaa2c41dbdcae093 not found: ID does not exist" Jul 17 19:30:47.223715 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.223684 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:30:47.229617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:47.229593 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-45e4d318-openshift-default-86b6c94979-lgrwv"] Jul 17 19:30:48.905837 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:30:48.905782 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" path="/var/lib/kubelet/pods/cd190a2a-00d7-49c4-b9a1-a9995edaee1d/volumes" Jul 17 19:31:36.076852 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.076820 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:31:36.077296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077265 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.077296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077278 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.077296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077295 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.077398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077300 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.077398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077307 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="511854a0-0e19-45d2-8945-0090c5eef0e4" containerName="main" Jul 17 19:31:36.077398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077313 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="511854a0-0e19-45d2-8945-0090c5eef0e4" containerName="main" Jul 17 19:31:36.077398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077397 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="511854a0-0e19-45d2-8945-0090c5eef0e4" containerName="main" Jul 17 19:31:36.077524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077407 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.077524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.077415 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="cd190a2a-00d7-49c4-b9a1-a9995edaee1d" containerName="istio-proxy" Jul 17 19:31:36.080551 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.080534 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.083250 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.083226 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-1f5606-openshift-default-dockercfg-c4qzh\"" Jul 17 19:31:36.092218 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.092196 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:31:36.163231 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163194 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163434 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163249 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163434 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163338 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9fm9\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163434 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163411 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163464 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163518 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163538 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163738 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163568 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.163738 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.163624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.264866 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.264831 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.264877 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.264896 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.264940 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.264983 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265033 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265061 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265397 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265121 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9fm9\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265397 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265323 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265397 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265368 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265624 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265594 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265957 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265630 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.265957 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.265675 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.266147 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.266117 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.267429 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.267402 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.267757 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.267737 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.277149 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.277129 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.277254 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.277158 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9fm9\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9\") pod \"e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.390857 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.390756 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:36.532849 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.532817 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:31:36.534866 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:31:36.534843 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode49b9e09_5a42_46aa_af91_038c0351c851.slice/crio-0c437bd2e9d60399d3c3b575513077e2838daa63d29435a32dcabf2ed67d1279 WatchSource:0}: Error finding container 0c437bd2e9d60399d3c3b575513077e2838daa63d29435a32dcabf2ed67d1279: Status 404 returned error can't find the container with id 0c437bd2e9d60399d3c3b575513077e2838daa63d29435a32dcabf2ed67d1279 Jul 17 19:31:36.537398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.537368 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:36.537470 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.537432 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:36.537470 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:36.537463 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:37.440579 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:37.440540 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerStarted","Data":"146bdb1957409be17d81e15f12f75f61c83fec3a96f8732b68339668044139b6"} Jul 17 19:31:37.441085 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:37.440595 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerStarted","Data":"0c437bd2e9d60399d3c3b575513077e2838daa63d29435a32dcabf2ed67d1279"} Jul 17 19:31:37.462748 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:37.462700 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podStartSLOduration=1.462684701 podStartE2EDuration="1.462684701s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:31:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:31:37.462556534 +0000 UTC m=+2179.113936990" watchObservedRunningTime="2026-07-17 19:31:37.462684701 +0000 UTC m=+2179.114065156" Jul 17 19:31:38.391871 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:38.391828 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:38.396708 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:38.396684 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:38.445617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:38.445578 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:38.446603 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:38.446581 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:44.457172 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.457136 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:31:44.460790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.460768 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.463698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.463677 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-eaec93-openshift-default-dockercfg-77dbk\"" Jul 17 19:31:44.470359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.470333 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:31:44.548463 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548423 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548661 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548492 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548661 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548576 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548661 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548624 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548821 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548672 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548821 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548702 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548821 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548735 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548821 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.548951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.548863 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh525\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.649574 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649535 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.649833 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649609 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.649833 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649693 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.649833 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649720 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.649833 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649750 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649866 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649945 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.649979 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh525\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650042 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650231 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650063 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650231 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650205 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650312 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650244 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650382 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650365 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.650448 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.650431 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.652507 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.652478 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.652609 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.652546 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.658115 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.658085 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.658211 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.658131 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh525\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525\") pod \"e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.776696 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.776586 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:44.922677 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.922633 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:31:44.924268 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:31:44.924245 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc8da44a8_9376_4f9e_81b4_23c2ba8c894d.slice/crio-0e155be7fe8ef2d15c4e0c7548d3311e113bd8f46de0c35c96cb096c7f980c7d WatchSource:0}: Error finding container 0e155be7fe8ef2d15c4e0c7548d3311e113bd8f46de0c35c96cb096c7f980c7d: Status 404 returned error can't find the container with id 0e155be7fe8ef2d15c4e0c7548d3311e113bd8f46de0c35c96cb096c7f980c7d Jul 17 19:31:44.926554 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.926516 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:44.926677 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.926605 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:44.926752 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:44.926705 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:45.484969 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:45.484923 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerStarted","Data":"9becfeacb914b30f6dd92301b7c047c8c13d8863b2ddc1799d308ceb36f23eb6"} Jul 17 19:31:45.485347 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:45.485021 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerStarted","Data":"0e155be7fe8ef2d15c4e0c7548d3311e113bd8f46de0c35c96cb096c7f980c7d"} Jul 17 19:31:45.507070 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:45.507009 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podStartSLOduration=1.506990258 podStartE2EDuration="1.506990258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:31:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:31:45.505300126 +0000 UTC m=+2187.156680595" watchObservedRunningTime="2026-07-17 19:31:45.506990258 +0000 UTC m=+2187.158370713" Jul 17 19:31:45.777441 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:45.777337 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:46.782524 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:46.782491 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:47.494827 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:47.494790 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:47.495855 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:47.495833 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:31:54.487824 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:54.487778 2584 patch_prober.go:28] interesting pod/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 17 19:31:54.488277 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:54.487865 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 17 19:31:55.537088 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.537053 2584 generic.go:356] "Generic (PLEG): container finished" podID="e49b9e09-5a42-46aa-af91-038c0351c851" containerID="146bdb1957409be17d81e15f12f75f61c83fec3a96f8732b68339668044139b6" exitCode=0 Jul 17 19:31:55.537510 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.537126 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerDied","Data":"146bdb1957409be17d81e15f12f75f61c83fec3a96f8732b68339668044139b6"} Jul 17 19:31:55.537588 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.537570 2584 scope.go:117] "RemoveContainer" containerID="146bdb1957409be17d81e15f12f75f61c83fec3a96f8732b68339668044139b6" Jul 17 19:31:55.538735 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.538706 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:55.538797 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.538774 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:55.538833 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:55.538806 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:31:56.547171 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:56.547120 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerStarted","Data":"b408b6ac32abb3a80e37cd5920d61c1da0227703ecf58705f64cebcf4af0be9c"} Jul 17 19:31:57.391618 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:57.391576 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:31:57.391966 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:57.391927 2584 patch_prober.go:28] interesting pod/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 17 19:31:57.392103 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:57.391995 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 17 19:31:58.391452 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:58.391415 2584 patch_prober.go:28] interesting pod/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 17 19:31:58.391895 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:58.391471 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 17 19:31:59.391189 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:59.391150 2584 patch_prober.go:28] interesting pod/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 17 19:31:59.391383 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:31:59.391200 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 17 19:32:00.391810 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:00.391765 2584 patch_prober.go:28] interesting pod/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body= Jul 17 19:32:00.392209 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:00.391830 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" Jul 17 19:32:01.395033 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:01.394994 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:32:01.395456 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:01.395432 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:32:01.396286 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:01.396263 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:32:03.478129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.477621 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:32846->10.134.0.45:15021: read: connection reset by peer" start-of-body= Jul 17 19:32:03.478129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.477708 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": read tcp 10.134.0.2:32846->10.134.0.45:15021: read: connection reset by peer" Jul 17 19:32:03.593733 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.593698 2584 generic.go:356] "Generic (PLEG): container finished" podID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerID="9becfeacb914b30f6dd92301b7c047c8c13d8863b2ddc1799d308ceb36f23eb6" exitCode=0 Jul 17 19:32:03.593909 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.593778 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerDied","Data":"9becfeacb914b30f6dd92301b7c047c8c13d8863b2ddc1799d308ceb36f23eb6"} Jul 17 19:32:03.594249 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.594230 2584 scope.go:117] "RemoveContainer" containerID="9becfeacb914b30f6dd92301b7c047c8c13d8863b2ddc1799d308ceb36f23eb6" Jul 17 19:32:03.595296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.595269 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:32:03.595352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.595336 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:32:03.595401 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:03.595369 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:32:04.601162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:04.601128 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerStarted","Data":"0f07bb33b69982d5c6bf144c7a8680249412e638e2f41081c1932f86cc8cedaa"} Jul 17 19:32:04.777880 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:04.777776 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:32:04.778583 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:04.778552 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 17 19:32:04.778868 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:04.778828 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 17 19:32:05.777239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:05.777200 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 17 19:32:05.777636 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:05.777263 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 17 19:32:06.777071 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:06.777033 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 17 19:32:06.777338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:06.777091 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 17 19:32:07.777987 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:07.777941 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 17 19:32:07.778414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:07.778006 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 17 19:32:08.777854 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:08.777816 2584 patch_prober.go:28] interesting pod/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" start-of-body= Jul 17 19:32:08.778034 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:08.777879 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.45:15021/healthz/ready\": dial tcp 10.134.0.45:15021: connect: connection refused" Jul 17 19:32:09.781366 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:09.781335 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:32:09.781776 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:09.781609 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:32:09.782486 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:09.782467 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:32:21.017531 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.017488 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:21.022037 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.022015 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.026313 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.026289 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-eaec93\"/\"default-dockercfg-zzsjg\"" Jul 17 19:32:21.026479 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.026312 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-eaec93\"/\"test-model-eaec93-kserve-self-signed-certs\"" Jul 17 19:32:21.026479 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.026346 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-eaec93\"/\"openshift-service-ca.crt\"" Jul 17 19:32:21.026479 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.026312 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-eaec93\"/\"kube-root-ca.crt\"" Jul 17 19:32:21.030949 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.030926 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:21.153214 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153180 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.153214 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153227 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.153483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153252 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.153483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153350 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.153483 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcv26\" (UniqueName: \"kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.153636 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.153522 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.254620 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254573 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.254620 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254618 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcv26\" (UniqueName: \"kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.254866 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254719 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.254866 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254802 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.254866 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.255030 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.254877 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.255087 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.255033 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.255143 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.255113 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.255214 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.255196 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.257304 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.257271 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.257534 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.257517 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.264614 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.264595 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcv26\" (UniqueName: \"kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26\") pod \"test-model-eaec93-kserve-59c77987bf-6z2fx\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.334631 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.334533 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:21.478666 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.478619 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:21.480717 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:32:21.480689 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poded4cfbd1_d9ee_4a2d_ba73_a4c42fa07a19.slice/crio-366854ea7c1eeef3841336dee4f2eb0096d625b9a82fd4ef55c9adc16ae29216 WatchSource:0}: Error finding container 366854ea7c1eeef3841336dee4f2eb0096d625b9a82fd4ef55c9adc16ae29216: Status 404 returned error can't find the container with id 366854ea7c1eeef3841336dee4f2eb0096d625b9a82fd4ef55c9adc16ae29216 Jul 17 19:32:21.693601 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.693515 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" event={"ID":"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19","Type":"ContainerStarted","Data":"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013"} Jul 17 19:32:21.693601 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.693560 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" event={"ID":"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19","Type":"ContainerStarted","Data":"366854ea7c1eeef3841336dee4f2eb0096d625b9a82fd4ef55c9adc16ae29216"} Jul 17 19:32:21.717817 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:21.717760 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" podStartSLOduration=1.717734586 podStartE2EDuration="1.717734586s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:32:21.716991543 +0000 UTC m=+2223.368371997" watchObservedRunningTime="2026-07-17 19:32:21.717734586 +0000 UTC m=+2223.369115040" Jul 17 19:32:31.334947 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:31.334895 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:31.334947 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:31.334957 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:31.347441 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:31.347413 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:31.755277 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:31.755244 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:40.929827 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:40.929727 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:55.951577 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:55.951524 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" podUID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" containerName="main" containerID="cri-o://5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013" gracePeriod=45 Jul 17 19:32:56.194934 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.194911 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:56.317991 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.317888 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fcv26\" (UniqueName: \"kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.317991 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.317967 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.318222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318046 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.318222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318080 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.318222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318105 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.318222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318136 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm\") pod \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\" (UID: \"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19\") " Jul 17 19:32:56.318470 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318438 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:32:56.318541 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318521 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tmp-dir\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.318598 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318545 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache" (OuterVolumeSpecName: "model-cache") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:32:56.318683 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.318306 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home" (OuterVolumeSpecName: "home") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:32:56.320569 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.320530 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:32:56.320569 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.320530 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm" (OuterVolumeSpecName: "dshm") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:32:56.320840 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.320618 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26" (OuterVolumeSpecName: "kube-api-access-fcv26") pod "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" (UID: "ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19"). InnerVolumeSpecName "kube-api-access-fcv26". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:32:56.419975 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.419936 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-home\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.420150 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.419986 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-tls-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.420150 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.420001 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-dshm\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.420150 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.420014 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fcv26\" (UniqueName: \"kubernetes.io/projected/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-kube-api-access-fcv26\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.420150 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.420027 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19-model-cache\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:32:56.882472 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.882439 2584 generic.go:356] "Generic (PLEG): container finished" podID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" containerID="5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013" exitCode=0 Jul 17 19:32:56.882684 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.882495 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" Jul 17 19:32:56.882684 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.882520 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" event={"ID":"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19","Type":"ContainerDied","Data":"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013"} Jul 17 19:32:56.882684 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.882566 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx" event={"ID":"ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19","Type":"ContainerDied","Data":"366854ea7c1eeef3841336dee4f2eb0096d625b9a82fd4ef55c9adc16ae29216"} Jul 17 19:32:56.882684 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.882589 2584 scope.go:117] "RemoveContainer" containerID="5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013" Jul 17 19:32:56.893509 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.893489 2584 scope.go:117] "RemoveContainer" containerID="5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013" Jul 17 19:32:56.894750 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:32:56.894613 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013\": container with ID starting with 5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013 not found: ID does not exist" containerID="5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013" Jul 17 19:32:56.894750 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.894680 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013"} err="failed to get container status \"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013\": rpc error: code = NotFound desc = could not find container \"5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013\": container with ID starting with 5bb5394a84dccc0f2ed33a2a64797ed683a28df0af0cb32da4a1d9a1e0ef1013 not found: ID does not exist" Jul 17 19:32:56.905497 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.905468 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:56.911613 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:56.911588 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-eaec93/test-model-eaec93-kserve-59c77987bf-6z2fx"] Jul 17 19:32:58.904907 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:32:58.904878 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" path="/var/lib/kubelet/pods/ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19/volumes" Jul 17 19:33:06.802088 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:06.802050 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:33:06.802546 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:06.802263 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" containerID="cri-o://b408b6ac32abb3a80e37cd5920d61c1da0227703ecf58705f64cebcf4af0be9c" gracePeriod=30 Jul 17 19:33:11.961423 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:11.961389 2584 generic.go:356] "Generic (PLEG): container finished" podID="e49b9e09-5a42-46aa-af91-038c0351c851" containerID="b408b6ac32abb3a80e37cd5920d61c1da0227703ecf58705f64cebcf4af0be9c" exitCode=0 Jul 17 19:33:11.961878 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:11.961465 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerDied","Data":"b408b6ac32abb3a80e37cd5920d61c1da0227703ecf58705f64cebcf4af0be9c"} Jul 17 19:33:11.961878 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:11.961529 2584 scope.go:117] "RemoveContainer" containerID="146bdb1957409be17d81e15f12f75f61c83fec3a96f8732b68339668044139b6" Jul 17 19:33:12.071214 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.071145 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:33:12.173570 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173534 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173804 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173597 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173804 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173638 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173804 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173707 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173988 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173836 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173988 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173880 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173988 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173917 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9fm9\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173988 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173945 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.173988 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.173978 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket\") pod \"e49b9e09-5a42-46aa-af91-038c0351c851\" (UID: \"e49b9e09-5a42-46aa-af91-038c0351c851\") " Jul 17 19:33:12.174229 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174069 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:12.174229 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174155 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:33:12.174325 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174278 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:12.174459 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174434 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:12.174674 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174609 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e49b9e09-5a42-46aa-af91-038c0351c851-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.174674 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174629 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.174674 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174663 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.175090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.174696 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data" (OuterVolumeSpecName: "istio-data") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:12.175982 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.175960 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token" (OuterVolumeSpecName: "istio-token") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:33:12.176327 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.176311 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9" (OuterVolumeSpecName: "kube-api-access-b9fm9") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "kube-api-access-b9fm9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:33:12.176493 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.176472 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:12.176493 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.176480 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e49b9e09-5a42-46aa-af91-038c0351c851" (UID: "e49b9e09-5a42-46aa-af91-038c0351c851"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:33:12.275372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275331 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e49b9e09-5a42-46aa-af91-038c0351c851-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.275372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275368 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9fm9\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-kube-api-access-b9fm9\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.275372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275380 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.275614 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275388 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.275614 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275398 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e49b9e09-5a42-46aa-af91-038c0351c851-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.275614 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.275405 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e49b9e09-5a42-46aa-af91-038c0351c851-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:12.967442 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.967413 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" Jul 17 19:33:12.967928 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.967458 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh" event={"ID":"e49b9e09-5a42-46aa-af91-038c0351c851","Type":"ContainerDied","Data":"0c437bd2e9d60399d3c3b575513077e2838daa63d29435a32dcabf2ed67d1279"} Jul 17 19:33:12.967928 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.967518 2584 scope.go:117] "RemoveContainer" containerID="b408b6ac32abb3a80e37cd5920d61c1da0227703ecf58705f64cebcf4af0be9c" Jul 17 19:33:12.994432 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.994388 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:33:12.994630 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:12.994460 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-1f5606-openshift-default-7bb8f96445-rn9mh"] Jul 17 19:33:14.905083 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:14.905049 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" path="/var/lib/kubelet/pods/e49b9e09-5a42-46aa-af91-038c0351c851/volumes" Jul 17 19:33:32.919313 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:32.919271 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:33:32.919790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:32.919546 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" containerID="cri-o://0f07bb33b69982d5c6bf144c7a8680249412e638e2f41081c1932f86cc8cedaa" gracePeriod=30 Jul 17 19:33:33.762008 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.761969 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:33:33.762476 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762451 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.762658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762506 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.762658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762524 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" containerName="main" Jul 17 19:33:33.762658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762532 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" containerName="main" Jul 17 19:33:33.762658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762549 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.762658 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762557 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.762916 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762715 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="ed4cfbd1-d9ee-4a2d-ba73-a4c42fa07a19" containerName="main" Jul 17 19:33:33.762916 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762729 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.762916 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.762738 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="e49b9e09-5a42-46aa-af91-038c0351c851" containerName="istio-proxy" Jul 17 19:33:33.765911 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.765889 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.768629 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.768612 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-0bfe89-openshift-default-dockercfg-7vt56\"" Jul 17 19:33:33.778319 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.778293 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:33:33.795527 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795489 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795706 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795592 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795706 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795670 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795706 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795699 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npvzs\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795856 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795754 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795856 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795784 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795856 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795815 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795971 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795882 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.795971 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.795934 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896398 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896353 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896597 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896411 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896597 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896435 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896597 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896452 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npvzs\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896597 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896483 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896597 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896574 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896908 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896625 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896908 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896702 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.896908 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.896758 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.897129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.897012 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.897129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.897111 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.897230 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.897164 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.897372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.897351 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.897420 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.897386 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.899078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.899050 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.899222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.899206 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.904832 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.904801 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:33.904943 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:33.904902 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npvzs\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs\") pod \"e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:34.076453 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:34.076346 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:34.226070 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:34.226044 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:33:34.228788 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:33:34.228754 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dca6ea5_0964_4181_8d4d_76849b7f4d55.slice/crio-3c2ab7e262e897b6ecfdeb676b059b7d435bd5962e4f92b58a03f34c8d782d1e WatchSource:0}: Error finding container 3c2ab7e262e897b6ecfdeb676b059b7d435bd5962e4f92b58a03f34c8d782d1e: Status 404 returned error can't find the container with id 3c2ab7e262e897b6ecfdeb676b059b7d435bd5962e4f92b58a03f34c8d782d1e Jul 17 19:33:34.231116 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:34.231077 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:34.231198 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:34.231159 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:34.231255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:34.231237 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:35.075291 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:35.075247 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerStarted","Data":"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f"} Jul 17 19:33:35.075485 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:35.075308 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerStarted","Data":"3c2ab7e262e897b6ecfdeb676b059b7d435bd5962e4f92b58a03f34c8d782d1e"} Jul 17 19:33:35.076556 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:35.076539 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:35.081353 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:35.081328 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:35.098446 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:35.098386 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podStartSLOduration=2.098369681 podStartE2EDuration="2.098369681s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:33:35.095978493 +0000 UTC m=+2296.747358950" watchObservedRunningTime="2026-07-17 19:33:35.098369681 +0000 UTC m=+2296.749750226" Jul 17 19:33:36.080380 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:36.080347 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:36.081480 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:36.081456 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:38.092869 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.092831 2584 generic.go:356] "Generic (PLEG): container finished" podID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerID="0f07bb33b69982d5c6bf144c7a8680249412e638e2f41081c1932f86cc8cedaa" exitCode=0 Jul 17 19:33:38.093248 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.092899 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerDied","Data":"0f07bb33b69982d5c6bf144c7a8680249412e638e2f41081c1932f86cc8cedaa"} Jul 17 19:33:38.093248 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.092959 2584 scope.go:117] "RemoveContainer" containerID="9becfeacb914b30f6dd92301b7c047c8c13d8863b2ddc1799d308ceb36f23eb6" Jul 17 19:33:38.184998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.184932 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:33:38.240960 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.240931 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.240986 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zh525\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241013 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241038 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241063 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241114 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241143 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241208 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241240 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data\") pod \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\" (UID: \"c8da44a8-9376-4f9e-81b4-23c2ba8c894d\") " Jul 17 19:33:38.241478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241246 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:38.241478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241335 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:38.241478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241375 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:38.241745 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241638 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:33:38.241745 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241714 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.241745 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241736 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.241894 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.241759 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.242602 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.242575 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data" (OuterVolumeSpecName: "istio-data") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:38.243832 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.243802 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525" (OuterVolumeSpecName: "kube-api-access-zh525") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "kube-api-access-zh525". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:33:38.244129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.244100 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token" (OuterVolumeSpecName: "istio-token") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:33:38.244316 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.244296 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:33:38.244626 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.244595 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c8da44a8-9376-4f9e-81b4-23c2ba8c894d" (UID: "c8da44a8-9376-4f9e-81b4-23c2ba8c894d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:33:38.343043 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343005 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.343043 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343039 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zh525\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-kube-api-access-zh525\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.343043 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343049 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.343291 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343058 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.343291 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343066 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:38.343291 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:38.343074 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c8da44a8-9376-4f9e-81b4-23c2ba8c894d-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:33:39.099547 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:39.099518 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" Jul 17 19:33:39.100026 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:39.099519 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d" event={"ID":"c8da44a8-9376-4f9e-81b4-23c2ba8c894d","Type":"ContainerDied","Data":"0e155be7fe8ef2d15c4e0c7548d3311e113bd8f46de0c35c96cb096c7f980c7d"} Jul 17 19:33:39.100026 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:39.099666 2584 scope.go:117] "RemoveContainer" containerID="0f07bb33b69982d5c6bf144c7a8680249412e638e2f41081c1932f86cc8cedaa" Jul 17 19:33:39.121565 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:39.121532 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:33:39.125349 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:39.125322 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-eaec93-openshift-default-64d7468659-cb65d"] Jul 17 19:33:40.905509 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:40.905472 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" path="/var/lib/kubelet/pods/c8da44a8-9376-4f9e-81b4-23c2ba8c894d/volumes" Jul 17 19:33:50.178921 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.178888 2584 generic.go:356] "Generic (PLEG): container finished" podID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerID="c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f" exitCode=0 Jul 17 19:33:50.179465 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.178939 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerDied","Data":"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f"} Jul 17 19:33:50.179533 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.179484 2584 scope.go:117] "RemoveContainer" containerID="c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f" Jul 17 19:33:50.180529 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.180496 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:50.180623 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.180565 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:50.180623 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:50.180598 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:33:51.080736 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:51.080696 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:51.185412 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:51.185376 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerStarted","Data":"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e"} Jul 17 19:33:52.077407 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:52.077361 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:52.078569 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:52.078545 2584 patch_prober.go:28] interesting pod/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 17 19:33:52.078655 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:52.078612 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 17 19:33:53.076772 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:53.076736 2584 patch_prober.go:28] interesting pod/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 17 19:33:53.077188 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:53.076792 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 17 19:33:54.077067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:54.077021 2584 patch_prober.go:28] interesting pod/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" start-of-body= Jul 17 19:33:54.077490 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:54.077090 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.47:15021/healthz/ready\": dial tcp 10.134.0.47:15021: connect: connection refused" Jul 17 19:33:55.081138 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:55.081101 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:55.081515 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:55.081387 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:33:55.082074 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:33:55.082055 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:34:23.631959 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:23.631925 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:34:23.632393 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:23.632143 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" containerID="cri-o://82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e" gracePeriod=30 Jul 17 19:34:25.084538 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:25.084495 2584 patch_prober.go:28] interesting pod/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 17 19:34:25.084966 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:25.084568 2584 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 17 19:34:28.896994 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:28.896925 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:34:29.090432 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090392 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090458 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090479 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090508 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090528 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090673 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090576 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090936 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090790 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:34:29.090993 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090952 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-npvzs\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.090993 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.090973 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:34:29.091109 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091069 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.091167 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091113 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket\") pod \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\" (UID: \"6dca6ea5-0964-4181-8d4d-76849b7f4d55\") " Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091342 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091393 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091669 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091691 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091709 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.091766 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091723 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.092172 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.091817 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data" (OuterVolumeSpecName: "istio-data") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:34:29.093124 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.093086 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:34:29.093283 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.093251 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:34:29.093460 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.093437 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token" (OuterVolumeSpecName: "istio-token") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:34:29.093460 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.093450 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs" (OuterVolumeSpecName: "kube-api-access-npvzs") pod "6dca6ea5-0964-4181-8d4d-76849b7f4d55" (UID: "6dca6ea5-0964-4181-8d4d-76849b7f4d55"). InnerVolumeSpecName "kube-api-access-npvzs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:34:29.193235 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.193144 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.193235 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.193177 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.193235 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.193186 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.193235 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.193194 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6dca6ea5-0964-4181-8d4d-76849b7f4d55-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.193235 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.193204 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-npvzs\" (UniqueName: \"kubernetes.io/projected/6dca6ea5-0964-4181-8d4d-76849b7f4d55-kube-api-access-npvzs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:34:29.381533 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.381498 2584 generic.go:356] "Generic (PLEG): container finished" podID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerID="82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e" exitCode=0 Jul 17 19:34:29.381758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.381550 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" Jul 17 19:34:29.381758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.381580 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerDied","Data":"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e"} Jul 17 19:34:29.381758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.381657 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl" event={"ID":"6dca6ea5-0964-4181-8d4d-76849b7f4d55","Type":"ContainerDied","Data":"3c2ab7e262e897b6ecfdeb676b059b7d435bd5962e4f92b58a03f34c8d782d1e"} Jul 17 19:34:29.381758 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.381676 2584 scope.go:117] "RemoveContainer" containerID="82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e" Jul 17 19:34:29.391465 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.391447 2584 scope.go:117] "RemoveContainer" containerID="c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f" Jul 17 19:34:29.399851 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.399833 2584 scope.go:117] "RemoveContainer" containerID="82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e" Jul 17 19:34:29.400103 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:34:29.400084 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e\": container with ID starting with 82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e not found: ID does not exist" containerID="82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e" Jul 17 19:34:29.400163 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.400112 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e"} err="failed to get container status \"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e\": rpc error: code = NotFound desc = could not find container \"82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e\": container with ID starting with 82c77fb2f1a56303992da34ec56fe1966912160bc8dbb9e555aff8031900312e not found: ID does not exist" Jul 17 19:34:29.400163 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.400130 2584 scope.go:117] "RemoveContainer" containerID="c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f" Jul 17 19:34:29.400344 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:34:29.400330 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f\": container with ID starting with c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f not found: ID does not exist" containerID="c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f" Jul 17 19:34:29.400384 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.400347 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f"} err="failed to get container status \"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f\": rpc error: code = NotFound desc = could not find container \"c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f\": container with ID starting with c09562349c1e21ca8fce61d8efed0f950023382bb3fb26e64090af6ad001b26f not found: ID does not exist" Jul 17 19:34:29.403813 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.403788 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:34:29.407875 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:29.407849 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-0bfe89-openshift-default-57dc7786bc-z74rl"] Jul 17 19:34:30.905102 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:30.905071 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" path="/var/lib/kubelet/pods/6dca6ea5-0964-4181-8d4d-76849b7f4d55/volumes" Jul 17 19:34:50.621616 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.621578 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:34:50.622221 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622200 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.622305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622224 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.622305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622258 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.622305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622268 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.622305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622285 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622305 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622294 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622315 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622323 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622420 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622434 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="c8da44a8-9376-4f9e-81b4-23c2ba8c894d" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622444 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.622592 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.622453 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="6dca6ea5-0964-4181-8d4d-76849b7f4d55" containerName="istio-proxy" Jul 17 19:34:50.627245 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.627223 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.630680 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.630633 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-b3dd7f-openshift-default-dockercfg-8fssw\"" Jul 17 19:34:50.637828 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.637803 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:34:50.697165 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697128 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697165 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697169 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697196 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697217 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697313 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697364 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697422 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697407 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xkfr\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697613 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697438 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.697613 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.697526 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798815 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798777 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798986 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798821 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798986 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798887 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798986 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798920 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798986 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798953 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5xkfr\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.798986 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.798982 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799069 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799110 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799137 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799315 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799503 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799451 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799503 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799479 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.799622 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.799607 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.800071 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.800048 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.801402 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.801375 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.801574 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.801555 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.808370 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.808338 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.808459 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.808446 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xkfr\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr\") pod \"e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:50.938816 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:50.938727 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:51.079799 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.079774 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:34:51.081948 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:34:51.081922 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod25c7f451_45af_4020_8bef_573fcf37c6b1.slice/crio-5ae118df8c7fca15ab319aa9a834ee964b6edbbae5ecb4631a1ec93ed2516805 WatchSource:0}: Error finding container 5ae118df8c7fca15ab319aa9a834ee964b6edbbae5ecb4631a1ec93ed2516805: Status 404 returned error can't find the container with id 5ae118df8c7fca15ab319aa9a834ee964b6edbbae5ecb4631a1ec93ed2516805 Jul 17 19:34:51.084271 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.084233 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:51.084345 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.084328 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:51.084397 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.084381 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:51.498590 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.498555 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerStarted","Data":"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851"} Jul 17 19:34:51.498790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.498610 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerStarted","Data":"5ae118df8c7fca15ab319aa9a834ee964b6edbbae5ecb4631a1ec93ed2516805"} Jul 17 19:34:51.520856 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.520804 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podStartSLOduration=1.520786519 podStartE2EDuration="1.520786519s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:34:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:34:51.51767324 +0000 UTC m=+2373.169053687" watchObservedRunningTime="2026-07-17 19:34:51.520786519 +0000 UTC m=+2373.172166969" Jul 17 19:34:51.939686 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:51.939562 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:52.943854 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:52.943824 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:53.511555 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:53.511522 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:53.512403 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:53.512377 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:34:58.456919 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.456883 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:34:58.461261 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.461236 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.464255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.464228 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-79d51b-openshift-default-dockercfg-gp6gh\"" Jul 17 19:34:58.471394 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.471368 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:34:58.577999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.577967 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dbzd\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.577999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578007 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578224 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578025 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578224 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578078 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578224 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578121 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578224 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578195 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578240 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578301 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.578359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.578319 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679761 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679725 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679771 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679824 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679845 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679894 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dbzd\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679922 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.679951 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679938 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.679967 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680015 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680375 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680320 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680428 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680373 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680467 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680423 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680557 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680539 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.680600 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.680564 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.682573 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.682541 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.682573 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.682561 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.688310 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.688290 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.688472 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.688456 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dbzd\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd\") pod \"e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.774025 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.773947 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:58.934212 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.934182 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:34:58.935919 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:34:58.935895 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod294d2fcd_f31f_46f4_b5c2_d45bf465c568.slice/crio-7b4ba4463472cac4cf74731221f7ad4f0c4368207df33e1ef808bd8ee77dd4bc WatchSource:0}: Error finding container 7b4ba4463472cac4cf74731221f7ad4f0c4368207df33e1ef808bd8ee77dd4bc: Status 404 returned error can't find the container with id 7b4ba4463472cac4cf74731221f7ad4f0c4368207df33e1ef808bd8ee77dd4bc Jul 17 19:34:58.938112 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.938082 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:58.938183 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.938145 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:58.938183 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:58.938174 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:34:59.543818 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:59.543785 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerStarted","Data":"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9"} Jul 17 19:34:59.544261 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:59.543838 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerStarted","Data":"7b4ba4463472cac4cf74731221f7ad4f0c4368207df33e1ef808bd8ee77dd4bc"} Jul 17 19:34:59.565624 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:59.565563 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podStartSLOduration=1.5655445719999999 podStartE2EDuration="1.565544572s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:34:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:34:59.564391401 +0000 UTC m=+2381.215771886" watchObservedRunningTime="2026-07-17 19:34:59.565544572 +0000 UTC m=+2381.216925029" Jul 17 19:34:59.774678 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:59.774580 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:34:59.779451 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:34:59.779421 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:00.548894 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:00.548856 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:00.549880 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:00.549857 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:07.585667 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.585614 2584 generic.go:356] "Generic (PLEG): container finished" podID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerID="2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851" exitCode=0 Jul 17 19:35:07.586140 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.585686 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerDied","Data":"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851"} Jul 17 19:35:07.586140 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.586096 2584 scope.go:117] "RemoveContainer" containerID="2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851" Jul 17 19:35:07.587090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.587061 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:07.587196 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.587130 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:07.587196 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:07.587165 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:08.511981 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:08.511946 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:35:08.596984 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:08.596950 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerStarted","Data":"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18"} Jul 17 19:35:08.941148 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:08.941046 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:35:08.941514 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:08.941428 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 17 19:35:08.941744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:08.941708 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 17 19:35:09.939497 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:09.939435 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 17 19:35:09.940040 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:09.939555 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 17 19:35:10.939242 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:10.939205 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 17 19:35:10.939452 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:10.939272 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 17 19:35:11.939242 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:11.939206 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 17 19:35:11.940259 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:11.939853 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 17 19:35:12.940103 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:12.940062 2584 patch_prober.go:28] interesting pod/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" start-of-body= Jul 17 19:35:12.941157 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:12.940720 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.48:15021/healthz/ready\": dial tcp 10.134.0.48:15021: connect: connection refused" Jul 17 19:35:13.943954 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:13.943926 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:35:13.944440 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:13.944301 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:35:13.945090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:13.945075 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:35:14.000357 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.000326 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:35:14.003501 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.003475 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.006554 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.006528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"default-dockercfg-gv6zz\"" Jul 17 19:35:14.006713 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.006528 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"auth-test-model-b3dd7f-kserve-self-signed-certs\"" Jul 17 19:35:14.006713 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.006559 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"openshift-service-ca.crt\"" Jul 17 19:35:14.009687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.007138 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"kube-root-ca.crt\"" Jul 17 19:35:14.028667 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.015879 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:35:14.043928 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.043901 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.044067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.043962 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.044067 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.044045 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.044302 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.044282 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cnj9w\" (UniqueName: \"kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.044355 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.044326 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.044396 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.044384 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145126 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145044 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cnj9w\" (UniqueName: \"kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145126 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145086 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145324 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145278 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145402 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145385 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145471 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145455 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145530 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145494 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.145916 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145891 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.146018 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145921 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.146018 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.145860 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.147689 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.147658 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.147893 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.147877 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.154155 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.154136 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cnj9w\" (UniqueName: \"kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w\") pod \"auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.335996 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.335959 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:14.477633 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.477608 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:35:14.479519 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:35:14.479493 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9121ef86_38be_4248_9406_fedb303a5d31.slice/crio-78db7cc92ec558b9ae065e96f9b8df8505bcec860389a1316f2ae5d5b017f150 WatchSource:0}: Error finding container 78db7cc92ec558b9ae065e96f9b8df8505bcec860389a1316f2ae5d5b017f150: Status 404 returned error can't find the container with id 78db7cc92ec558b9ae065e96f9b8df8505bcec860389a1316f2ae5d5b017f150 Jul 17 19:35:14.634078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.633988 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" event={"ID":"9121ef86-38be-4248-9406-fedb303a5d31","Type":"ContainerStarted","Data":"0af33a80ac518c1e466d229e525a12d6d0f5683bfe701e0dc443ed3b186a6598"} Jul 17 19:35:14.634078 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.634032 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" event={"ID":"9121ef86-38be-4248-9406-fedb303a5d31","Type":"ContainerStarted","Data":"78db7cc92ec558b9ae065e96f9b8df8505bcec860389a1316f2ae5d5b017f150"} Jul 17 19:35:14.635554 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.635528 2584 generic.go:356] "Generic (PLEG): container finished" podID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerID="08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9" exitCode=0 Jul 17 19:35:14.635700 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.635602 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerDied","Data":"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9"} Jul 17 19:35:14.636113 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.636099 2584 scope.go:117] "RemoveContainer" containerID="08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9" Jul 17 19:35:14.637023 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.636998 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:14.637080 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.637067 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:14.637115 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.637099 2584 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892164Ki","pods":"250"} Jul 17 19:35:14.654950 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:14.654718 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" podStartSLOduration=1.654696796 podStartE2EDuration="1.654696796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:35:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:35:14.653958928 +0000 UTC m=+2396.305339392" watchObservedRunningTime="2026-07-17 19:35:14.654696796 +0000 UTC m=+2396.306077252" Jul 17 19:35:15.549744 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:15.549688 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:15.646100 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:15.646062 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerStarted","Data":"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409"} Jul 17 19:35:15.774932 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:15.774901 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:15.776050 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:15.776026 2584 patch_prober.go:28] interesting pod/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 17 19:35:15.776178 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:15.776072 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 17 19:35:16.774530 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:16.774496 2584 patch_prober.go:28] interesting pod/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 17 19:35:16.775026 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:16.774556 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 17 19:35:17.775244 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:17.775207 2584 patch_prober.go:28] interesting pod/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 17 19:35:17.775759 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:17.775271 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 17 19:35:18.774449 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:18.774410 2584 patch_prober.go:28] interesting pod/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" start-of-body= Jul 17 19:35:18.774621 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:18.774510 2584 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.49:15021/healthz/ready\": dial tcp 10.134.0.49:15021: connect: connection refused" Jul 17 19:35:19.778686 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:19.778628 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:19.779090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:19.778919 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:19.779637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:19.779619 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:35:24.337051 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:24.337012 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:24.337523 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:24.337069 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:24.350607 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:24.350578 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:35:24.703316 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:35:24.703289 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:37:41.284286 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:41.284206 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:37:56.305293 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.305252 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" podUID="9121ef86-38be-4248-9406-fedb303a5d31" containerName="main" containerID="cri-o://0af33a80ac518c1e466d229e525a12d6d0f5683bfe701e0dc443ed3b186a6598" gracePeriod=45 Jul 17 19:37:56.494146 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.494111 2584 generic.go:356] "Generic (PLEG): container finished" podID="9121ef86-38be-4248-9406-fedb303a5d31" containerID="0af33a80ac518c1e466d229e525a12d6d0f5683bfe701e0dc443ed3b186a6598" exitCode=0 Jul 17 19:37:56.494337 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.494191 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" event={"ID":"9121ef86-38be-4248-9406-fedb303a5d31","Type":"ContainerDied","Data":"0af33a80ac518c1e466d229e525a12d6d0f5683bfe701e0dc443ed3b186a6598"} Jul 17 19:37:56.578339 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.578308 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:37:56.641458 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641419 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.641699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641494 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cnj9w\" (UniqueName: \"kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.641699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641518 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.641699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641534 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.641699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641573 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.641699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641626 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs\") pod \"9121ef86-38be-4248-9406-fedb303a5d31\" (UID: \"9121ef86-38be-4248-9406-fedb303a5d31\") " Jul 17 19:37:56.642004 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641826 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home" (OuterVolumeSpecName: "home") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:37:56.642071 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.641998 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache" (OuterVolumeSpecName: "model-cache") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:37:56.642071 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.642015 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:37:56.642186 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.642168 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-home\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:56.642241 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.642193 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-tmp-dir\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:56.642241 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.642210 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-model-cache\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:56.644251 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.644215 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w" (OuterVolumeSpecName: "kube-api-access-cnj9w") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "kube-api-access-cnj9w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:37:56.644513 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.644487 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:37:56.644513 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.644490 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm" (OuterVolumeSpecName: "dshm") pod "9121ef86-38be-4248-9406-fedb303a5d31" (UID: "9121ef86-38be-4248-9406-fedb303a5d31"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:37:56.742940 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.742895 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9121ef86-38be-4248-9406-fedb303a5d31-tls-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:56.742940 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.742932 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9121ef86-38be-4248-9406-fedb303a5d31-dshm\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:56.742940 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:56.742943 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cnj9w\" (UniqueName: \"kubernetes.io/projected/9121ef86-38be-4248-9406-fedb303a5d31-kube-api-access-cnj9w\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:37:57.501887 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:57.501849 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" Jul 17 19:37:57.501887 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:57.501857 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd" event={"ID":"9121ef86-38be-4248-9406-fedb303a5d31","Type":"ContainerDied","Data":"78db7cc92ec558b9ae065e96f9b8df8505bcec860389a1316f2ae5d5b017f150"} Jul 17 19:37:57.502439 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:57.501910 2584 scope.go:117] "RemoveContainer" containerID="0af33a80ac518c1e466d229e525a12d6d0f5683bfe701e0dc443ed3b186a6598" Jul 17 19:37:57.519362 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:57.519328 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:37:57.525637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:57.525600 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/auth-test-model-b3dd7f-kserve-6bdf75f6b7-gbmmd"] Jul 17 19:37:58.905665 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:37:58.905613 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9121ef86-38be-4248-9406-fedb303a5d31" path="/var/lib/kubelet/pods/9121ef86-38be-4248-9406-fedb303a5d31/volumes" Jul 17 19:38:18.714614 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.714577 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:38:18.715265 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.715243 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9121ef86-38be-4248-9406-fedb303a5d31" containerName="main" Jul 17 19:38:18.715320 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.715270 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="9121ef86-38be-4248-9406-fedb303a5d31" containerName="main" Jul 17 19:38:18.715372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.715361 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="9121ef86-38be-4248-9406-fedb303a5d31" containerName="main" Jul 17 19:38:18.719825 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.719803 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.724300 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.724260 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-79d51b\"/\"kube-root-ca.crt\"" Jul 17 19:38:18.724478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.724374 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-79d51b\"/\"openshift-service-ca.crt\"" Jul 17 19:38:18.724478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.724260 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-79d51b\"/\"sub-test-model-79d51b-kserve-self-signed-certs\"" Jul 17 19:38:18.724686 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.724635 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-79d51b\"/\"default-dockercfg-sx4s7\"" Jul 17 19:38:18.727063 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.727030 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:38:18.876898 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.876852 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.876898 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.876897 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.877260 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.876932 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.877260 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.876978 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.877260 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.876998 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.877260 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.877129 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbb7s\" (UniqueName: \"kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978350 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978245 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978350 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978332 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978577 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978386 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978577 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978426 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978577 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978507 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978776 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978621 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cbb7s\" (UniqueName: \"kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978843 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978782 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.978896 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.978880 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.979287 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.979265 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.980829 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.980804 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.981187 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.981169 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:18.988090 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:18.988059 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbb7s\" (UniqueName: \"kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s\") pod \"sub-test-model-79d51b-kserve-597b5b847-5jb9c\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:19.032338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:19.032306 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:19.177939 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:19.177904 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:38:19.181996 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:38:19.181965 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd7e1405a_63d1_4798_82a0_8c71f3a25c39.slice/crio-9ad64c79c1a415b2271d2216bf9191cb171614d4fff79c05285ac77a773cccf0 WatchSource:0}: Error finding container 9ad64c79c1a415b2271d2216bf9191cb171614d4fff79c05285ac77a773cccf0: Status 404 returned error can't find the container with id 9ad64c79c1a415b2271d2216bf9191cb171614d4fff79c05285ac77a773cccf0 Jul 17 19:38:19.628813 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:19.628707 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" event={"ID":"d7e1405a-63d1-4798-82a0-8c71f3a25c39","Type":"ContainerStarted","Data":"e25f2965c8f131fa38f35b61e81ab1d2bcda5efa34789dc789e5c1e1bad1e040"} Jul 17 19:38:19.628813 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:19.628763 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" event={"ID":"d7e1405a-63d1-4798-82a0-8c71f3a25c39","Type":"ContainerStarted","Data":"9ad64c79c1a415b2271d2216bf9191cb171614d4fff79c05285ac77a773cccf0"} Jul 17 19:38:19.650118 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:19.650061 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" podStartSLOduration=1.650042223 podStartE2EDuration="1.650042223s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:38:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:38:19.648044804 +0000 UTC m=+2581.299425260" watchObservedRunningTime="2026-07-17 19:38:19.650042223 +0000 UTC m=+2581.301422725" Jul 17 19:38:29.032704 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:29.032665 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:29.033177 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:29.032906 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:29.045666 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:29.045623 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:38:29.693794 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:38:29.693764 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:39:06.721019 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:06.720981 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:39:16.808999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.808962 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:39:16.812955 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.812932 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.815861 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.815833 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"openshift-service-ca.crt\"" Jul 17 19:39:16.816008 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.815834 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"kube-root-ca.crt\"" Jul 17 19:39:16.817359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.817336 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"default-dockercfg-gv6zz\"" Jul 17 19:39:16.817687 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.817366 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-b3dd7f\"/\"rate-test-model-b3dd7f-kserve-self-signed-certs\"" Jul 17 19:39:16.820702 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.820680 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:39:16.871279 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871235 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5ffg\" (UniqueName: \"kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.871279 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871287 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.871508 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871350 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.871508 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871374 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.871508 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871413 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.871508 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.871453 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972107 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972071 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972153 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5ffg\" (UniqueName: \"kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972296 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972202 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972429 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972315 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972429 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972393 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972540 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972467 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972704 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972678 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.972994 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.972880 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.973081 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.973040 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.974861 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.974837 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.974948 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.974838 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:16.980875 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:16.980853 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5ffg\" (UniqueName: \"kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg\") pod \"rate-test-model-b3dd7f-kserve-684884b7f5-trb8r\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:17.125866 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:17.125750 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:17.271126 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:17.271100 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:39:17.273552 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:39:17.273521 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf58bd182_d1c2_4173_ac76_63ad8b390fff.slice/crio-de61bd0b8b1934815db4321fa21402980c95d947e936688ad877fed6fc7e4a96 WatchSource:0}: Error finding container de61bd0b8b1934815db4321fa21402980c95d947e936688ad877fed6fc7e4a96: Status 404 returned error can't find the container with id de61bd0b8b1934815db4321fa21402980c95d947e936688ad877fed6fc7e4a96 Jul 17 19:39:17.938692 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:17.938638 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" event={"ID":"f58bd182-d1c2-4173-ac76-63ad8b390fff","Type":"ContainerStarted","Data":"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5"} Jul 17 19:39:17.938692 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:17.938699 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" event={"ID":"f58bd182-d1c2-4173-ac76-63ad8b390fff","Type":"ContainerStarted","Data":"de61bd0b8b1934815db4321fa21402980c95d947e936688ad877fed6fc7e4a96"} Jul 17 19:39:17.960006 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:17.959954 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" podStartSLOduration=1.959937212 podStartE2EDuration="1.959937212s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:39:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:39:17.956750243 +0000 UTC m=+2639.608130724" watchObservedRunningTime="2026-07-17 19:39:17.959937212 +0000 UTC m=+2639.611317667" Jul 17 19:39:21.744622 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:21.744567 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" podUID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" containerName="main" containerID="cri-o://e25f2965c8f131fa38f35b61e81ab1d2bcda5efa34789dc789e5c1e1bad1e040" gracePeriod=45 Jul 17 19:39:21.965656 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:21.965618 2584 generic.go:356] "Generic (PLEG): container finished" podID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" containerID="e25f2965c8f131fa38f35b61e81ab1d2bcda5efa34789dc789e5c1e1bad1e040" exitCode=0 Jul 17 19:39:21.965822 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:21.965698 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" event={"ID":"d7e1405a-63d1-4798-82a0-8c71f3a25c39","Type":"ContainerDied","Data":"e25f2965c8f131fa38f35b61e81ab1d2bcda5efa34789dc789e5c1e1bad1e040"} Jul 17 19:39:22.007733 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.007664 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:39:22.128224 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128186 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128251 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128294 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128340 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cbb7s\" (UniqueName: \"kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128414 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128370 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128638 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128471 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir\") pod \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\" (UID: \"d7e1405a-63d1-4798-82a0-8c71f3a25c39\") " Jul 17 19:39:22.128638 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128578 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home" (OuterVolumeSpecName: "home") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:39:22.128638 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128591 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache" (OuterVolumeSpecName: "model-cache") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:39:22.128844 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128812 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:39:22.128981 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128957 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-model-cache\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.128981 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128983 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-home\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.129162 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.128996 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tmp-dir\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.130869 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.130841 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm" (OuterVolumeSpecName: "dshm") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:39:22.130869 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.130861 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s" (OuterVolumeSpecName: "kube-api-access-cbb7s") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "kube-api-access-cbb7s". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:39:22.131011 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.130862 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d7e1405a-63d1-4798-82a0-8c71f3a25c39" (UID: "d7e1405a-63d1-4798-82a0-8c71f3a25c39"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:39:22.229965 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.229926 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cbb7s\" (UniqueName: \"kubernetes.io/projected/d7e1405a-63d1-4798-82a0-8c71f3a25c39-kube-api-access-cbb7s\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.229965 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.229958 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d7e1405a-63d1-4798-82a0-8c71f3a25c39-tls-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.229965 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.229969 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d7e1405a-63d1-4798-82a0-8c71f3a25c39-dshm\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:39:22.973214 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.973183 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" Jul 17 19:39:22.973636 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.973219 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c" event={"ID":"d7e1405a-63d1-4798-82a0-8c71f3a25c39","Type":"ContainerDied","Data":"9ad64c79c1a415b2271d2216bf9191cb171614d4fff79c05285ac77a773cccf0"} Jul 17 19:39:22.973636 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.973284 2584 scope.go:117] "RemoveContainer" containerID="e25f2965c8f131fa38f35b61e81ab1d2bcda5efa34789dc789e5c1e1bad1e040" Jul 17 19:39:22.991911 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.991884 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:39:22.995501 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:22.995476 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-79d51b/sub-test-model-79d51b-kserve-597b5b847-5jb9c"] Jul 17 19:39:24.905190 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:24.905157 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" path="/var/lib/kubelet/pods/d7e1405a-63d1-4798-82a0-8c71f3a25c39/volumes" Jul 17 19:39:27.126571 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:27.126531 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:27.127073 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:27.126699 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:27.139075 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:27.139044 2584 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:39:28.011028 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:39:28.010997 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:40:29.945222 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:29.945189 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:40:44.967460 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:44.967406 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" podUID="f58bd182-d1c2-4173-ac76-63ad8b390fff" containerName="main" containerID="cri-o://53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5" gracePeriod=45 Jul 17 19:40:45.228246 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.228171 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:40:45.289860 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.289827 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5ffg\" (UniqueName: \"kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290063 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.289880 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290063 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.289913 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290063 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290010 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290133 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290190 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir\") pod \"f58bd182-d1c2-4173-ac76-63ad8b390fff\" (UID: \"f58bd182-d1c2-4173-ac76-63ad8b390fff\") " Jul 17 19:40:45.290356 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290296 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache" (OuterVolumeSpecName: "model-cache") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:40:45.290487 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290440 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home" (OuterVolumeSpecName: "home") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:40:45.290487 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290459 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:40:45.290600 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290577 2584 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-tmp-dir\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.290600 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290597 2584 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-model-cache\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.290899 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.290608 2584 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-home\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.292329 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.292303 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 17 19:40:45.292799 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.292767 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm" (OuterVolumeSpecName: "dshm") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:40:45.292799 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.292785 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg" (OuterVolumeSpecName: "kube-api-access-g5ffg") pod "f58bd182-d1c2-4173-ac76-63ad8b390fff" (UID: "f58bd182-d1c2-4173-ac76-63ad8b390fff"). InnerVolumeSpecName "kube-api-access-g5ffg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:40:45.391229 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.391188 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5ffg\" (UniqueName: \"kubernetes.io/projected/f58bd182-d1c2-4173-ac76-63ad8b390fff-kube-api-access-g5ffg\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.391229 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.391221 2584 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f58bd182-d1c2-4173-ac76-63ad8b390fff-tls-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.391229 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.391229 2584 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/f58bd182-d1c2-4173-ac76-63ad8b390fff-dshm\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:40:45.408492 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.408454 2584 generic.go:356] "Generic (PLEG): container finished" podID="f58bd182-d1c2-4173-ac76-63ad8b390fff" containerID="53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5" exitCode=0 Jul 17 19:40:45.408698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.408501 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" Jul 17 19:40:45.408698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.408499 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" event={"ID":"f58bd182-d1c2-4173-ac76-63ad8b390fff","Type":"ContainerDied","Data":"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5"} Jul 17 19:40:45.408698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.408635 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r" event={"ID":"f58bd182-d1c2-4173-ac76-63ad8b390fff","Type":"ContainerDied","Data":"de61bd0b8b1934815db4321fa21402980c95d947e936688ad877fed6fc7e4a96"} Jul 17 19:40:45.408698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.408685 2584 scope.go:117] "RemoveContainer" containerID="53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5" Jul 17 19:40:45.419476 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.419447 2584 scope.go:117] "RemoveContainer" containerID="53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5" Jul 17 19:40:45.419852 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:40:45.419829 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5\": container with ID starting with 53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5 not found: ID does not exist" containerID="53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5" Jul 17 19:40:45.419932 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.419863 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5"} err="failed to get container status \"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5\": rpc error: code = NotFound desc = could not find container \"53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5\": container with ID starting with 53f0d483a91f252ca82d18a3ff5676c53faa62854f7028b39c6c4d8d6437e7e5 not found: ID does not exist" Jul 17 19:40:45.430585 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.430547 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:40:45.433011 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:45.432985 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-b3dd7f/rate-test-model-b3dd7f-kserve-684884b7f5-trb8r"] Jul 17 19:40:46.905281 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:40:46.905244 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f58bd182-d1c2-4173-ac76-63ad8b390fff" path="/var/lib/kubelet/pods/f58bd182-d1c2-4173-ac76-63ad8b390fff/volumes" Jul 17 19:41:15.877790 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:15.877758 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:41:15.878384 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:15.878070 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" containerID="cri-o://dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18" gracePeriod=30 Jul 17 19:41:21.126311 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.126282 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:41:21.165492 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165403 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165492 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165441 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165492 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165486 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165541 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165624 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165727 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5xkfr\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165757 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165802 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.165879 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165837 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo\") pod \"25c7f451-45af-4020-8bef-573fcf37c6b1\" (UID: \"25c7f451-45af-4020-8bef-573fcf37c6b1\") " Jul 17 19:41:21.166194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165964 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:41:21.166194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.165974 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:21.166309 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.166204 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:21.166309 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.166220 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:21.166309 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.166228 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.166309 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.166264 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/25c7f451-45af-4020-8bef-573fcf37c6b1-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.166500 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.166376 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data" (OuterVolumeSpecName: "istio-data") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:21.168337 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.168306 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr" (OuterVolumeSpecName: "kube-api-access-5xkfr") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "kube-api-access-5xkfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:41:21.168460 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.168355 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:41:21.168525 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.168467 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:21.168588 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.168563 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token" (OuterVolumeSpecName: "istio-token") pod "25c7f451-45af-4020-8bef-573fcf37c6b1" (UID: "25c7f451-45af-4020-8bef-573fcf37c6b1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267560 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267605 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267615 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267623 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5xkfr\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-kube-api-access-5xkfr\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267633 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.267617 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267668 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/25c7f451-45af-4020-8bef-573fcf37c6b1-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.268026 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.267678 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/25c7f451-45af-4020-8bef-573fcf37c6b1-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:21.596982 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.596941 2584 generic.go:356] "Generic (PLEG): container finished" podID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerID="dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18" exitCode=0 Jul 17 19:41:21.597187 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.596994 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" Jul 17 19:41:21.597187 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.596995 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerDied","Data":"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18"} Jul 17 19:41:21.597187 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.597125 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq" event={"ID":"25c7f451-45af-4020-8bef-573fcf37c6b1","Type":"ContainerDied","Data":"5ae118df8c7fca15ab319aa9a834ee964b6edbbae5ecb4631a1ec93ed2516805"} Jul 17 19:41:21.597187 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.597152 2584 scope.go:117] "RemoveContainer" containerID="dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18" Jul 17 19:41:21.607227 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.607210 2584 scope.go:117] "RemoveContainer" containerID="2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851" Jul 17 19:41:21.615682 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.615636 2584 scope.go:117] "RemoveContainer" containerID="dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18" Jul 17 19:41:21.615967 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:41:21.615947 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18\": container with ID starting with dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18 not found: ID does not exist" containerID="dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18" Jul 17 19:41:21.616051 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.615978 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18"} err="failed to get container status \"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18\": rpc error: code = NotFound desc = could not find container \"dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18\": container with ID starting with dd841c0462ef02d97cf8a2ea9e9ac1503e965374efd46a9620206ab7eab0ce18 not found: ID does not exist" Jul 17 19:41:21.616051 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.616004 2584 scope.go:117] "RemoveContainer" containerID="2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851" Jul 17 19:41:21.616244 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:41:21.616228 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851\": container with ID starting with 2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851 not found: ID does not exist" containerID="2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851" Jul 17 19:41:21.616282 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.616251 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851"} err="failed to get container status \"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851\": rpc error: code = NotFound desc = could not find container \"2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851\": container with ID starting with 2ec7240aad45b7974e06cee8a45402b321115ef9705974a0c7d7215881e6e851 not found: ID does not exist" Jul 17 19:41:21.622815 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.622792 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:41:21.628580 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:21.628557 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-b3dd7f-openshift-default-644dfcbc8c-kflpq"] Jul 17 19:41:22.905129 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:22.905096 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" path="/var/lib/kubelet/pods/25c7f451-45af-4020-8bef-573fcf37c6b1/volumes" Jul 17 19:41:41.989511 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:41.989429 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:41:41.989977 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:41.989704 2584 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" containerID="cri-o://e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409" gracePeriod=30 Jul 17 19:41:47.244195 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.244168 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:41:47.341039 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.340941 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341039 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.340986 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341039 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341015 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341091 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341119 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341208 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341249 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341277 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:47.341338 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341283 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dbzd\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341361 2584 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo\") pod \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\" (UID: \"294d2fcd-f31f-46f4-b5c2-d45bf465c568\") " Jul 17 19:41:47.341637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341394 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:47.341637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341411 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:47.341637 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341437 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 17 19:41:47.341952 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341802 2584 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.341952 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341828 2584 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istiod-ca-cert\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.341952 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341868 2584 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-credential-socket\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.341952 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.341882 2584 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-workload-certs\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.342191 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.342170 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data" (OuterVolumeSpecName: "istio-data") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:47.344386 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.344359 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd" (OuterVolumeSpecName: "kube-api-access-5dbzd") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "kube-api-access-5dbzd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:41:47.344386 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.344370 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token" (OuterVolumeSpecName: "istio-token") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 17 19:41:47.344553 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.344359 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 17 19:41:47.344553 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.344365 2584 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "294d2fcd-f31f-46f4-b5c2-d45bf465c568" (UID: "294d2fcd-f31f-46f4-b5c2-d45bf465c568"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 17 19:41:47.443014 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.442977 2584 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-envoy\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.443014 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.443010 2584 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-token\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.443014 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.443018 2584 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-data\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.443014 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.443027 2584 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dbzd\" (UniqueName: \"kubernetes.io/projected/294d2fcd-f31f-46f4-b5c2-d45bf465c568-kube-api-access-5dbzd\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.443298 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.443037 2584 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/294d2fcd-f31f-46f4-b5c2-d45bf465c568-istio-podinfo\") on node \"ip-10-0-133-230.ec2.internal\" DevicePath \"\"" Jul 17 19:41:47.730670 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.730609 2584 generic.go:356] "Generic (PLEG): container finished" podID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerID="e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409" exitCode=0 Jul 17 19:41:47.730670 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.730670 2584 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" Jul 17 19:41:47.730904 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.730695 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerDied","Data":"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409"} Jul 17 19:41:47.730904 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.730783 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k" event={"ID":"294d2fcd-f31f-46f4-b5c2-d45bf465c568","Type":"ContainerDied","Data":"7b4ba4463472cac4cf74731221f7ad4f0c4368207df33e1ef808bd8ee77dd4bc"} Jul 17 19:41:47.730904 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.730807 2584 scope.go:117] "RemoveContainer" containerID="e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409" Jul 17 19:41:47.740846 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.740828 2584 scope.go:117] "RemoveContainer" containerID="08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9" Jul 17 19:41:47.749163 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.749143 2584 scope.go:117] "RemoveContainer" containerID="e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409" Jul 17 19:41:47.749428 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:41:47.749408 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409\": container with ID starting with e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409 not found: ID does not exist" containerID="e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409" Jul 17 19:41:47.749498 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.749442 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409"} err="failed to get container status \"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409\": rpc error: code = NotFound desc = could not find container \"e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409\": container with ID starting with e68005da07bfd9ac073c4069ef8a7df78271dec16d46181cc20836545a20d409 not found: ID does not exist" Jul 17 19:41:47.749498 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.749470 2584 scope.go:117] "RemoveContainer" containerID="08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9" Jul 17 19:41:47.749811 ip-10-0-133-230 kubenswrapper[2584]: E0717 19:41:47.749794 2584 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9\": container with ID starting with 08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9 not found: ID does not exist" containerID="08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9" Jul 17 19:41:47.749873 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.749818 2584 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9"} err="failed to get container status \"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9\": rpc error: code = NotFound desc = could not find container \"08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9\": container with ID starting with 08a24d6f87c8cc39f4f2a5a02ef44eabd2cd21fef8822da3a7e54bfa55c5f9a9 not found: ID does not exist" Jul 17 19:41:47.753538 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.753517 2584 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:41:47.757204 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:47.757182 2584 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-79d51b-openshift-default-7bf7956bff-svr9k"] Jul 17 19:41:48.397805 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:48.397775 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f99c964f4-qqnt2_cfd230f0-2476-4249-8c2a-2872725fa06a/manager/0.log" Jul 17 19:41:48.905423 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:48.905388 2584 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" path="/var/lib/kubelet/pods/294d2fcd-f31f-46f4-b5c2-d45bf465c568/volumes" Jul 17 19:41:49.024987 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:49.024958 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b55dc8c5f-9sv6h_559b0b1a-7216-46a5-810f-553605103653/copy-manifests/0.log" Jul 17 19:41:49.036570 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:49.036544 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b55dc8c5f-9sv6h_559b0b1a-7216-46a5-810f-553605103653/manager/0.log" Jul 17 19:41:52.715566 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:52.715535 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s7wcp_ba491637-6554-41a4-95a5-2189d500167f/kuadrant-console-plugin/0.log" Jul 17 19:41:52.958205 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:52.958167 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qsks5_b3c22461-3bf6-4fb4-baa1-a790081e1ab8/manager/0.log" Jul 17 19:41:53.562081 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:53.562050 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mrppc_280bdaa2-bcdf-42e6-a181-16da6358990d/istio-proxy/0.log" Jul 17 19:41:53.895019 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:53.894938 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ddf547cfd-mzwlq_d0291cde-e2ae-4207-b9b1-5b61a0eca94f/kube-auth-proxy/0.log" Jul 17 19:41:54.143999 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:54.143957 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xxtpk_d1138eed-e019-44fa-a13d-15e5c812fc46/payload-pre-processing/0.log" Jul 17 19:41:54.946764 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:54.946724 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-75b4d_ac6abfad-f4e4-4e9d-9930-545717f67ae9/main/0.log" Jul 17 19:41:55.278133 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:41:55.278100 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-5fc6fb456f-9bxcw_9e27b151-f35b-4732-8ac8-fcc91e93ddc0/main/0.log" Jul 17 19:42:01.153339 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153296 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mpmt/must-gather-j8pjk"] Jul 17 19:42:01.153915 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153896 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153918 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153931 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f58bd182-d1c2-4173-ac76-63ad8b390fff" containerName="main" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153940 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="f58bd182-d1c2-4173-ac76-63ad8b390fff" containerName="main" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153955 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153965 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153975 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.153998 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.153987 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154027 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" containerName="main" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154036 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" containerName="main" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154141 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154153 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="d7e1405a-63d1-4798-82a0-8c71f3a25c39" containerName="main" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154161 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="f58bd182-d1c2-4173-ac76-63ad8b390fff" containerName="main" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154170 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154178 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154188 2584 memory_manager.go:368] "RemoveStaleState removing state" podUID="294d2fcd-f31f-46f4-b5c2-d45bf465c568" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154300 2584 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.154352 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.154309 2584 state_mem.go:107] "Deleted CPUSet assignment" podUID="25c7f451-45af-4020-8bef-573fcf37c6b1" containerName="istio-proxy" Jul 17 19:42:01.157698 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.157675 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.160607 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.160587 2584 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8mpmt\"/\"default-dockercfg-zcdlp\"" Jul 17 19:42:01.162018 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.161991 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mpmt\"/\"openshift-service-ca.crt\"" Jul 17 19:42:01.162104 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.162000 2584 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8mpmt\"/\"kube-root-ca.crt\"" Jul 17 19:42:01.167372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.167347 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mpmt/must-gather-j8pjk"] Jul 17 19:42:01.284066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.284019 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rc95n\" (UniqueName: \"kubernetes.io/projected/e5246b45-2403-472b-a265-6842f7c2030f-kube-api-access-rc95n\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.284066 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.284069 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5246b45-2403-472b-a265-6842f7c2030f-must-gather-output\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.385657 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.385595 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rc95n\" (UniqueName: \"kubernetes.io/projected/e5246b45-2403-472b-a265-6842f7c2030f-kube-api-access-rc95n\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.385657 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.385664 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5246b45-2403-472b-a265-6842f7c2030f-must-gather-output\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.386020 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.386001 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/e5246b45-2403-472b-a265-6842f7c2030f-must-gather-output\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.394791 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.394752 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rc95n\" (UniqueName: \"kubernetes.io/projected/e5246b45-2403-472b-a265-6842f7c2030f-kube-api-access-rc95n\") pod \"must-gather-j8pjk\" (UID: \"e5246b45-2403-472b-a265-6842f7c2030f\") " pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.467845 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.467809 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" Jul 17 19:42:01.619490 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.619462 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mpmt/must-gather-j8pjk"] Jul 17 19:42:01.621340 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:42:01.621306 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode5246b45_2403_472b_a265_6842f7c2030f.slice/crio-63e25e44cc9526d2afc9d49bf875491b60764ddbeee8ffd5f1d8fe99151a754c WatchSource:0}: Error finding container 63e25e44cc9526d2afc9d49bf875491b60764ddbeee8ffd5f1d8fe99151a754c: Status 404 returned error can't find the container with id 63e25e44cc9526d2afc9d49bf875491b60764ddbeee8ffd5f1d8fe99151a754c Jul 17 19:42:01.623170 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.623149 2584 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 17 19:42:01.800801 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:01.800694 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" event={"ID":"e5246b45-2403-472b-a265-6842f7c2030f","Type":"ContainerStarted","Data":"63e25e44cc9526d2afc9d49bf875491b60764ddbeee8ffd5f1d8fe99151a754c"} Jul 17 19:42:02.809709 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:02.809584 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" event={"ID":"e5246b45-2403-472b-a265-6842f7c2030f","Type":"ContainerStarted","Data":"5604efae3357d5d9685937bac7a6447db9f982bce57674fd706401689f4dcb80"} Jul 17 19:42:02.809709 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:02.809637 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" event={"ID":"e5246b45-2403-472b-a265-6842f7c2030f","Type":"ContainerStarted","Data":"6b36053949487d1b809669af6f84bde6e0ec99a1c0bd471f7f168f1b7fc9044c"} Jul 17 19:42:02.827868 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:02.827800 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8mpmt/must-gather-j8pjk" podStartSLOduration=0.945145792 podStartE2EDuration="1.827778025s" totalImagesPullingTime="882.632233ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-17 19:42:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:42:02.825263484 +0000 UTC m=+2804.476643940" watchObservedRunningTime="2026-07-17 19:42:02.827778025 +0000 UTC m=+2804.479158481" Jul 17 19:42:04.089654 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:04.089604 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-f67qc_c014f0b8-1ba7-40e4-861a-c27dc7c871b0/global-pull-secret-syncer/0.log" Jul 17 19:42:04.314140 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:04.314103 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-8q9k6_d0a06bcd-e178-4db3-891d-ec3d90901d1e/konnectivity-agent/0.log" Jul 17 19:42:04.340581 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:04.340498 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-133-230.ec2.internal_50351eb0dda3c09fbc3ee35a82fae765/haproxy/0.log" Jul 17 19:42:09.850302 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:09.850266 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-s7wcp_ba491637-6554-41a4-95a5-2189d500167f/kuadrant-console-plugin/0.log" Jul 17 19:42:10.051514 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:10.051424 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-qsks5_b3c22461-3bf6-4fb4-baa1-a790081e1ab8/manager/0.log" Jul 17 19:42:11.915776 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:11.915673 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/alertmanager/0.log" Jul 17 19:42:11.942962 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:11.942927 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/config-reloader/0.log" Jul 17 19:42:11.965755 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:11.965713 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/kube-rbac-proxy-web/0.log" Jul 17 19:42:11.993544 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:11.993520 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/kube-rbac-proxy/0.log" Jul 17 19:42:12.018745 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.018707 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/kube-rbac-proxy-metric/0.log" Jul 17 19:42:12.046167 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.046134 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/prom-label-proxy/0.log" Jul 17 19:42:12.071101 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.071071 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_703b7e90-3d53-4ceb-bb0b-c2a35e3876ac/init-config-reloader/0.log" Jul 17 19:42:12.155119 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.155081 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-b7h29_08ec5972-0c66-407f-aff1-5e238be5106e/kube-state-metrics/0.log" Jul 17 19:42:12.173258 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.173175 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-b7h29_08ec5972-0c66-407f-aff1-5e238be5106e/kube-rbac-proxy-main/0.log" Jul 17 19:42:12.197099 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.197066 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-56c5594674-b7h29_08ec5972-0c66-407f-aff1-5e238be5106e/kube-rbac-proxy-self/0.log" Jul 17 19:42:12.225062 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.225015 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-55f8d9db6b-xkln2_b264b71e-1111-4220-8ba6-d1c75cbc2d55/metrics-server/0.log" Jul 17 19:42:12.250703 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.250676 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-7b7b7bc8bf-cjfzv_b7664783-2ba8-4c0a-826e-c13bbf19b76c/monitoring-plugin/0.log" Jul 17 19:42:12.438838 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.438757 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s57j5_1fbb2101-cc97-46e4-a085-5ca56a6a017f/node-exporter/0.log" Jul 17 19:42:12.459977 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.459945 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s57j5_1fbb2101-cc97-46e4-a085-5ca56a6a017f/kube-rbac-proxy/0.log" Jul 17 19:42:12.484834 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.484803 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-s57j5_1fbb2101-cc97-46e4-a085-5ca56a6a017f/init-textfile/0.log" Jul 17 19:42:12.512389 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.512357 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5zbdc_7bd52955-7d4b-4ef5-b42b-7808e526fa63/kube-rbac-proxy-main/0.log" Jul 17 19:42:12.534705 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.534675 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5zbdc_7bd52955-7d4b-4ef5-b42b-7808e526fa63/kube-rbac-proxy-self/0.log" Jul 17 19:42:12.562047 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.562017 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-79f6958cdf-5zbdc_7bd52955-7d4b-4ef5-b42b-7808e526fa63/openshift-state-metrics/0.log" Jul 17 19:42:12.597342 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.597314 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/prometheus/0.log" Jul 17 19:42:12.618993 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.618964 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/config-reloader/0.log" Jul 17 19:42:12.648088 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.648057 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/thanos-sidecar/0.log" Jul 17 19:42:12.675134 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.675100 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/kube-rbac-proxy-web/0.log" Jul 17 19:42:12.705386 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.705363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/kube-rbac-proxy/0.log" Jul 17 19:42:12.730582 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.730554 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/kube-rbac-proxy-thanos/0.log" Jul 17 19:42:12.751842 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.751813 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_d41ad0f7-bfed-4be6-af63-4c14d21ddc18/init-config-reloader/0.log" Jul 17 19:42:12.782609 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.782578 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-v6mcz_48281b7d-0e8f-42bc-9cef-149f27a842f8/prometheus-operator/0.log" Jul 17 19:42:12.801490 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.801459 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-5945b5cfd-v6mcz_48281b7d-0e8f-42bc-9cef-149f27a842f8/kube-rbac-proxy/0.log" Jul 17 19:42:12.861525 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.861492 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84fbf498b5-qlnlt_03728292-ebe0-46ae-991d-048ef0cd68a7/telemeter-client/0.log" Jul 17 19:42:12.885130 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.885102 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84fbf498b5-qlnlt_03728292-ebe0-46ae-991d-048ef0cd68a7/reload/0.log" Jul 17 19:42:12.911099 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.911068 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-84fbf498b5-qlnlt_03728292-ebe0-46ae-991d-048ef0cd68a7/kube-rbac-proxy/0.log" Jul 17 19:42:12.942328 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.942293 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/thanos-query/0.log" Jul 17 19:42:12.965441 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.965363 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/kube-rbac-proxy-web/0.log" Jul 17 19:42:12.985304 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:12.985264 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/kube-rbac-proxy/0.log" Jul 17 19:42:13.005726 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:13.005685 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/prom-label-proxy/0.log" Jul 17 19:42:13.045143 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:13.045103 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/kube-rbac-proxy-rules/0.log" Jul 17 19:42:13.066797 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:13.066762 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d69ff8d68-tl4qk_013525df-e413-4025-8ffb-d39f2f8b1daa/kube-rbac-proxy-metrics/0.log" Jul 17 19:42:14.578738 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.578695 2584 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn"] Jul 17 19:42:14.585372 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.585340 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.592712 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.592681 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn"] Jul 17 19:42:14.657040 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.657002 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-podres\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.657226 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.657060 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-lib-modules\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.657226 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.657123 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-proc\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.657226 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.657184 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-sys\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.657362 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.657242 2584 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6c84t\" (UniqueName: \"kubernetes.io/projected/75b53b69-a3a5-4c07-8392-a74180eb4b9e-kube-api-access-6c84t\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758720 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758688 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-lib-modules\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758929 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758750 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-proc\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758929 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758806 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-sys\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758929 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758842 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-proc\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758929 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758868 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6c84t\" (UniqueName: \"kubernetes.io/projected/75b53b69-a3a5-4c07-8392-a74180eb4b9e-kube-api-access-6c84t\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.758929 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758893 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-lib-modules\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.759213 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758931 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-sys\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.759213 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.758947 2584 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-podres\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.759213 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.759051 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/75b53b69-a3a5-4c07-8392-a74180eb4b9e-podres\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.770013 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.769980 2584 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6c84t\" (UniqueName: \"kubernetes.io/projected/75b53b69-a3a5-4c07-8392-a74180eb4b9e-kube-api-access-6c84t\") pod \"perf-node-gather-daemonset-tmnvn\" (UID: \"75b53b69-a3a5-4c07-8392-a74180eb4b9e\") " pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:14.903512 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:14.903408 2584 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:15.113726 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.113695 2584 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn"] Jul 17 19:42:15.116234 ip-10-0-133-230 kubenswrapper[2584]: W0717 19:42:15.116201 2584 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod75b53b69_a3a5_4c07_8392_a74180eb4b9e.slice/crio-193a8bd362f6b950bcf8d010ab6f0eeff3e99a36338cd611066561ad836013a2 WatchSource:0}: Error finding container 193a8bd362f6b950bcf8d010ab6f0eeff3e99a36338cd611066561ad836013a2: Status 404 returned error can't find the container with id 193a8bd362f6b950bcf8d010ab6f0eeff3e99a36338cd611066561ad836013a2 Jul 17 19:42:15.495357 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.495318 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8cc7cdf69-57snr_9640041c-9183-48db-b0c6-dd0cc9c192e2/console/0.log" Jul 17 19:42:15.913354 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.913251 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" event={"ID":"75b53b69-a3a5-4c07-8392-a74180eb4b9e","Type":"ContainerStarted","Data":"05cc32bfdf97eb80738ee103f2d7ad4b20312e64f76e2288b91b567b9677529b"} Jul 17 19:42:15.914007 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.913980 2584 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" event={"ID":"75b53b69-a3a5-4c07-8392-a74180eb4b9e","Type":"ContainerStarted","Data":"193a8bd362f6b950bcf8d010ab6f0eeff3e99a36338cd611066561ad836013a2"} Jul 17 19:42:15.914178 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.914162 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:15.934241 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:15.934170 2584 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" podStartSLOduration=1.934147603 podStartE2EDuration="1.934147603s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-17 19:42:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-17 19:42:15.93258586 +0000 UTC m=+2817.583966315" watchObservedRunningTime="2026-07-17 19:42:15.934147603 +0000 UTC m=+2817.585528059" Jul 17 19:42:17.226194 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:17.226162 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vvzl_78bf9e23-a025-4b34-8199-9a733270ac55/dns/0.log" Jul 17 19:42:17.248385 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:17.248352 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2vvzl_78bf9e23-a025-4b34-8199-9a733270ac55/kube-rbac-proxy/0.log" Jul 17 19:42:17.383520 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:17.383465 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-h9bzt_dbcb13f3-7841-445c-a433-5234b674aeca/dns-node-resolver/0.log" Jul 17 19:42:18.081002 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:18.080969 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-jhjxm_792eb6c8-4a63-4882-8985-10b8bcf3a1b9/node-ca/0.log" Jul 17 19:42:19.082860 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:19.082826 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mrppc_280bdaa2-bcdf-42e6-a181-16da6358990d/istio-proxy/0.log" Jul 17 19:42:19.251384 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:19.251349 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ddf547cfd-mzwlq_d0291cde-e2ae-4207-b9b1-5b61a0eca94f/kube-auth-proxy/0.log" Jul 17 19:42:19.389478 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:19.389393 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xxtpk_d1138eed-e019-44fa-a13d-15e5c812fc46/payload-pre-processing/0.log" Jul 17 19:42:20.222852 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:20.222817 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-s47l6_c887fff7-76a2-4122-94ee-18a2ba272cde/serve-healthcheck-canary/0.log" Jul 17 19:42:20.905699 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:20.905667 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmkdl_73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d/kube-rbac-proxy/0.log" Jul 17 19:42:20.926239 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:20.926213 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmkdl_73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d/exporter/0.log" Jul 17 19:42:20.953931 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:20.953900 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-tmkdl_73ac3ab2-6bb5-42e0-ab4a-b80949ac9a1d/extractor/0.log" Jul 17 19:42:22.938255 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:22.938228 2584 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8mpmt/perf-node-gather-daemonset-tmnvn" Jul 17 19:42:23.489218 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:23.489163 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-6f99c964f4-qqnt2_cfd230f0-2476-4249-8c2a-2872725fa06a/manager/0.log" Jul 17 19:42:23.832201 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:23.832105 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b55dc8c5f-9sv6h_559b0b1a-7216-46a5-810f-553605103653/manager/0.log" Jul 17 19:42:23.853359 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:23.853328 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-b55dc8c5f-9sv6h_559b0b1a-7216-46a5-810f-553605103653/copy-manifests/0.log" Jul 17 19:42:24.904709 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:24.904678 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mrppc_280bdaa2-bcdf-42e6-a181-16da6358990d/istio-proxy/0.log" Jul 17 19:42:25.053237 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:25.053154 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ddf547cfd-mzwlq_d0291cde-e2ae-4207-b9b1-5b61a0eca94f/kube-auth-proxy/0.log" Jul 17 19:42:25.163895 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:25.163865 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xxtpk_d1138eed-e019-44fa-a13d-15e5c812fc46/payload-pre-processing/0.log" Jul 17 19:42:25.218286 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:25.218255 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-64d7f9f4cf-d2vms_4944bffe-e2d1-4dee-93d2-bcfb5f64a5a3/manager/0.log" Jul 17 19:42:25.289756 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:25.289726 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-9t6ft_4b508fa1-c46b-4ace-970c-94ff9d045075/openshift-lws-operator/0.log" Jul 17 19:42:26.509015 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:26.508979 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-6576859cf4mrppc_280bdaa2-bcdf-42e6-a181-16da6358990d/istio-proxy/0.log" Jul 17 19:42:26.674572 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:26.674543 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-ddf547cfd-mzwlq_d0291cde-e2ae-4207-b9b1-5b61a0eca94f/kube-auth-proxy/0.log" Jul 17 19:42:26.791351 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:26.791253 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7677b774bb-xxtpk_d1138eed-e019-44fa-a13d-15e5c812fc46/payload-pre-processing/0.log" Jul 17 19:42:32.299485 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.299452 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/kube-multus-additional-cni-plugins/0.log" Jul 17 19:42:32.322623 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.322589 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/egress-router-binary-copy/0.log" Jul 17 19:42:32.344504 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.344480 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/cni-plugins/0.log" Jul 17 19:42:32.367854 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.367823 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/bond-cni-plugin/0.log" Jul 17 19:42:32.389477 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.389445 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/routeoverride-cni/0.log" Jul 17 19:42:32.412551 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.412520 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/whereabouts-cni-bincopy/0.log" Jul 17 19:42:32.434138 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.434104 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-vn44d_f8f76668-6cf7-48a3-88dc-998738fa44c4/whereabouts-cni/0.log" Jul 17 19:42:32.553561 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.553476 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-rpcgw_aa36ca8f-467b-4d37-8eae-125febadac44/kube-multus/0.log" Jul 17 19:42:32.684798 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.684765 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lc7dh_1b173869-5333-421b-8a0f-f8dfc9a72faf/network-metrics-daemon/0.log" Jul 17 19:42:32.704545 ip-10-0-133-230 kubenswrapper[2584]: I0717 19:42:32.704510 2584 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lc7dh_1b173869-5333-421b-8a0f-f8dfc9a72faf/kube-rbac-proxy/0.log"