Jul 24 12:19:03.615501 ip-10-0-132-126 systemd[1]: Starting Kubernetes Kubelet... Jul 24 12:19:03.935127 ip-10-0-132-126 kubenswrapper[2578]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 12:19:03.935127 ip-10-0-132-126 kubenswrapper[2578]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 24 12:19:03.935127 ip-10-0-132-126 kubenswrapper[2578]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 12:19:03.935127 ip-10-0-132-126 kubenswrapper[2578]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 24 12:19:03.935127 ip-10-0-132-126 kubenswrapper[2578]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 24 12:19:03.937070 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.936943 2578 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 24 12:19:03.942881 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942750 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.942881 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942877 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.942881 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942883 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.942881 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942888 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942892 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942896 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942899 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942902 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942905 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942908 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942911 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942914 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942916 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942919 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942922 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942925 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942927 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942930 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942933 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942936 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942938 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942941 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942944 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.943071 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942947 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942949 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942952 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942954 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942957 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942959 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942961 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942964 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942966 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942970 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942972 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942975 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942978 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942980 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942986 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942989 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942992 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942995 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.942997 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943000 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.943558 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943002 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943005 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943007 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943010 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943014 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943017 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943020 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943022 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943025 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943028 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943031 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943034 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943037 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943042 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943045 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943049 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943051 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943054 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943056 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943059 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.944025 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943061 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943064 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943066 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943069 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943071 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943074 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943078 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943082 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943084 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943087 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943089 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943092 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943094 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943099 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943101 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943104 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943106 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943109 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943112 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943115 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.944499 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943118 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943121 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943123 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943126 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943128 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943130 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943133 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943136 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943138 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943141 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943144 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943147 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943149 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943151 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943154 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943156 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943915 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943924 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943929 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.944953 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943932 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943935 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943938 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943941 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943943 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943946 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943948 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943951 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943954 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943956 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943959 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943962 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943964 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943967 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943970 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943973 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943975 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943978 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943980 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.945398 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943983 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943985 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943987 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943989 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943992 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943995 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943997 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.943999 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944002 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944004 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944007 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944009 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944013 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944016 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944018 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944020 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944023 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944025 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944029 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944031 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.945847 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944036 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944039 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944043 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944045 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944048 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944050 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944054 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944057 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944061 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944063 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944066 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944069 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944071 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944074 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944076 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944079 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944081 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944084 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944086 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944089 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.946333 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944091 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944094 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944097 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944099 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944102 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944104 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944107 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944109 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944111 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944114 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944116 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944118 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944121 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944123 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944125 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944128 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944130 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944132 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944135 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944137 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.946813 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944139 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944143 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944145 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944147 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944149 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944153 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944156 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944158 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944161 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944163 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944166 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944168 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944170 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944173 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944175 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944180 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944183 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944626 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944633 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.947287 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944637 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944641 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944644 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944647 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944649 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944652 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944655 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944658 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944660 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944663 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944666 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944668 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944671 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944673 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944676 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944679 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944682 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944684 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944686 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944689 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.947745 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944691 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944693 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944696 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944698 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944701 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944704 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944706 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944709 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944712 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944714 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944717 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944720 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944722 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944724 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944727 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944729 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944732 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944734 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944737 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944739 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.948211 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944742 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944744 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944746 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944749 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944751 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944753 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944756 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944758 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944761 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944763 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944765 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944768 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944770 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944772 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944775 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944777 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944779 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944782 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944784 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944787 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.948675 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944790 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944793 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944795 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944798 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944800 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944803 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944805 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944809 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944813 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944816 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944818 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944820 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944823 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944825 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944828 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944830 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944832 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944835 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944837 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.949175 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944839 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944842 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944845 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944847 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944849 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944851 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944854 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944856 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944858 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944861 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944863 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944866 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944870 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944874 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944877 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944879 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944882 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.944884 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944956 2578 flags.go:64] FLAG: --address="0.0.0.0" Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944963 2578 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944969 2578 flags.go:64] FLAG: --anonymous-auth="true" Jul 24 12:19:03.949642 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944973 2578 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944977 2578 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944980 2578 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944984 2578 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944988 2578 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944991 2578 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944994 2578 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.944997 2578 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945001 2578 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945003 2578 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945006 2578 flags.go:64] FLAG: --cgroup-root="" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945010 2578 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945013 2578 flags.go:64] FLAG: --client-ca-file="" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945016 2578 flags.go:64] FLAG: --cloud-provider="external" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945019 2578 flags.go:64] FLAG: --cluster-dns="[]" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945024 2578 flags.go:64] FLAG: --cluster-domain="" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945027 2578 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945030 2578 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945033 2578 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945038 2578 flags.go:64] FLAG: --container-log-max-files="5" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945042 2578 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945045 2578 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945047 2578 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 24 12:19:03.950119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945051 2578 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945054 2578 flags.go:64] FLAG: --contention-profiling="false" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945056 2578 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945059 2578 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945062 2578 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945065 2578 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945068 2578 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945071 2578 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945074 2578 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945076 2578 flags.go:64] FLAG: --enable-load-reader="false" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945079 2578 flags.go:64] FLAG: --enable-server="true" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945082 2578 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945087 2578 flags.go:64] FLAG: --event-burst="100" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945094 2578 flags.go:64] FLAG: --event-qps="50" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945096 2578 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945099 2578 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945102 2578 flags.go:64] FLAG: --eviction-hard="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945106 2578 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945109 2578 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945112 2578 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945115 2578 flags.go:64] FLAG: --eviction-soft="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945118 2578 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945120 2578 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945124 2578 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945127 2578 flags.go:64] FLAG: --experimental-mounter-path="" Jul 24 12:19:03.950705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945129 2578 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945132 2578 flags.go:64] FLAG: --fail-swap-on="true" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945135 2578 flags.go:64] FLAG: --feature-gates="" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945141 2578 flags.go:64] FLAG: --file-check-frequency="20s" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945144 2578 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945146 2578 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945149 2578 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945152 2578 flags.go:64] FLAG: --healthz-port="10248" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945156 2578 flags.go:64] FLAG: --help="false" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945159 2578 flags.go:64] FLAG: --hostname-override="ip-10-0-132-126.ec2.internal" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945162 2578 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945165 2578 flags.go:64] FLAG: --http-check-frequency="20s" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945168 2578 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945171 2578 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945174 2578 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945177 2578 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945180 2578 flags.go:64] FLAG: --image-service-endpoint="" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945182 2578 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945185 2578 flags.go:64] FLAG: --kube-api-burst="100" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945202 2578 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945205 2578 flags.go:64] FLAG: --kube-api-qps="50" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945208 2578 flags.go:64] FLAG: --kube-reserved="" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945211 2578 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945214 2578 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 24 12:19:03.951381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945217 2578 flags.go:64] FLAG: --kubelet-cgroups="" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945219 2578 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945222 2578 flags.go:64] FLAG: --lock-file="" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945225 2578 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945228 2578 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945231 2578 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945236 2578 flags.go:64] FLAG: --log-json-split-stream="false" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945240 2578 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945242 2578 flags.go:64] FLAG: --log-text-split-stream="false" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945245 2578 flags.go:64] FLAG: --logging-format="text" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945248 2578 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945253 2578 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945255 2578 flags.go:64] FLAG: --manifest-url="" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945258 2578 flags.go:64] FLAG: --manifest-url-header="" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945262 2578 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945265 2578 flags.go:64] FLAG: --max-open-files="1000000" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945270 2578 flags.go:64] FLAG: --max-pods="110" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945273 2578 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945276 2578 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945279 2578 flags.go:64] FLAG: --memory-manager-policy="None" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945282 2578 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945285 2578 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945288 2578 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945291 2578 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945298 2578 flags.go:64] FLAG: --node-status-max-images="50" Jul 24 12:19:03.951960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945301 2578 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945303 2578 flags.go:64] FLAG: --oom-score-adj="-999" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945306 2578 flags.go:64] FLAG: --pod-cidr="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945309 2578 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:7505336a406394b8c1c77d14d890a9f628aa4cfff172312179cb9af8c28c30bb" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945314 2578 flags.go:64] FLAG: --pod-manifest-path="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945317 2578 flags.go:64] FLAG: --pod-max-pids="-1" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945320 2578 flags.go:64] FLAG: --pods-per-core="0" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945323 2578 flags.go:64] FLAG: --port="10250" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945325 2578 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945328 2578 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0892a21b17c42a353" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945331 2578 flags.go:64] FLAG: --qos-reserved="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945334 2578 flags.go:64] FLAG: --read-only-port="10255" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945337 2578 flags.go:64] FLAG: --register-node="true" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945339 2578 flags.go:64] FLAG: --register-with-taints="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945343 2578 flags.go:64] FLAG: --registry-burst="10" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945347 2578 flags.go:64] FLAG: --registry-qps="5" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945350 2578 flags.go:64] FLAG: --reserved-cpus="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945352 2578 flags.go:64] FLAG: --reserved-memory="" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945356 2578 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945361 2578 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945364 2578 flags.go:64] FLAG: --rotate-certificates="false" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945367 2578 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945370 2578 flags.go:64] FLAG: --runonce="false" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945373 2578 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945377 2578 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 24 12:19:03.952573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945379 2578 flags.go:64] FLAG: --seccomp-default="false" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945382 2578 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945385 2578 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945388 2578 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945390 2578 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945393 2578 flags.go:64] FLAG: --storage-driver-password="root" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945396 2578 flags.go:64] FLAG: --storage-driver-secure="false" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945399 2578 flags.go:64] FLAG: --storage-driver-table="stats" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945402 2578 flags.go:64] FLAG: --storage-driver-user="root" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945404 2578 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945407 2578 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945410 2578 flags.go:64] FLAG: --system-cgroups="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945413 2578 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945418 2578 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945420 2578 flags.go:64] FLAG: --tls-cert-file="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945423 2578 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945428 2578 flags.go:64] FLAG: --tls-min-version="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945430 2578 flags.go:64] FLAG: --tls-private-key-file="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945433 2578 flags.go:64] FLAG: --topology-manager-policy="none" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945435 2578 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945438 2578 flags.go:64] FLAG: --topology-manager-scope="container" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945441 2578 flags.go:64] FLAG: --v="2" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945444 2578 flags.go:64] FLAG: --version="false" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945449 2578 flags.go:64] FLAG: --vmodule="" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945452 2578 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 24 12:19:03.953162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945455 2578 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945567 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945570 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945573 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945577 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945580 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945583 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945586 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945589 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945591 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945594 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945597 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945600 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945604 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945607 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945610 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945613 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945615 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945618 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.953877 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945620 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945622 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945624 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945627 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945629 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945632 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945634 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945636 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945639 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945641 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945644 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945646 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945649 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945652 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945655 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945657 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945660 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945662 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945666 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.954500 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945670 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945673 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945675 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945678 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945680 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945683 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945685 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945688 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945690 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945692 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945695 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945698 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945700 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945702 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945705 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945707 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945709 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945712 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945714 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945717 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.954965 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945719 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945721 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945724 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945726 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945729 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945731 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945734 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945738 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945740 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945743 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945745 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945749 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945751 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945753 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945757 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945760 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945763 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945765 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945768 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945770 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.955447 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945772 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945775 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945778 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945780 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945782 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945785 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945787 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945789 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945792 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945794 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945797 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945799 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945801 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945804 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945806 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945809 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945811 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945813 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945816 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945820 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.955921 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945822 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.945825 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.945832 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.952979 2578 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.952996 2578 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.953598 2578 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.953607 2578 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953663 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953669 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953672 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953675 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953680 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953683 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953687 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.956408 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953689 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953692 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953695 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953698 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953701 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953703 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953706 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953709 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953711 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953714 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953716 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953719 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953722 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953724 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953726 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953729 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953732 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953734 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953736 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953739 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.956808 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953741 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953744 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953747 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953749 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953752 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953754 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953756 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953759 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953761 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953764 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953766 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953768 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953771 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953773 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953776 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953778 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953781 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953783 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953785 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953788 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.957296 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953791 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953793 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953796 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953798 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953801 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953803 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953807 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953811 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953813 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953816 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953818 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953820 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953823 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953825 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953828 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953831 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953833 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953835 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953838 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953840 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.957795 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953843 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953845 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953847 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953849 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953852 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953854 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953856 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953859 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953861 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953864 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953867 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953870 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953873 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953875 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953878 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953880 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953883 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953887 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953889 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953892 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.958273 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953895 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953897 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953900 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953902 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953905 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953907 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953910 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953912 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953915 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953917 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953920 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.953924 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.953929 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954035 2578 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 24 12:19:03.958755 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954038 2578 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954042 2578 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954046 2578 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954049 2578 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954052 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954055 2578 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954057 2578 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954060 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954063 2578 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954066 2578 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954068 2578 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954071 2578 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954073 2578 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954076 2578 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954079 2578 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954081 2578 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954084 2578 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954086 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954088 2578 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 24 12:19:03.959079 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954091 2578 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954093 2578 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954095 2578 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954098 2578 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954100 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954103 2578 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954105 2578 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954107 2578 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954111 2578 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954113 2578 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954115 2578 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954118 2578 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954120 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954122 2578 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954125 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954127 2578 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954129 2578 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954132 2578 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954134 2578 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954136 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 24 12:19:03.959539 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954139 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954141 2578 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954143 2578 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954146 2578 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954148 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954151 2578 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954153 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954155 2578 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954158 2578 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954160 2578 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954164 2578 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954167 2578 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954170 2578 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954172 2578 feature_gate.go:328] unrecognized feature gate: Example Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954174 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954177 2578 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954179 2578 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954181 2578 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954184 2578 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 24 12:19:03.960008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954186 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954204 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954209 2578 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954212 2578 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954215 2578 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954219 2578 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954222 2578 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954225 2578 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954227 2578 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954229 2578 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954231 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954234 2578 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954236 2578 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954239 2578 feature_gate.go:328] unrecognized feature gate: Example2 Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954241 2578 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954243 2578 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954246 2578 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954248 2578 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954251 2578 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954253 2578 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 24 12:19:03.960484 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954256 2578 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954259 2578 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954261 2578 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954264 2578 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954266 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954269 2578 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954271 2578 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954275 2578 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954278 2578 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954280 2578 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954283 2578 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954285 2578 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954287 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954290 2578 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954292 2578 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954297 2578 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954299 2578 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954301 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954304 2578 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 24 12:19:03.960948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:03.954306 2578 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.954311 2578 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.954427 2578 server.go:962] "Client rotation is on, will bootstrap in background" Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.956414 2578 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.957563 2578 server.go:1019] "Starting client certificate rotation" Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.957662 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.957695 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 24 12:19:03.961417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.959650 2578 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 12:19:03.964611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.964585 2578 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 24 12:19:03.973687 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.973665 2578 log.go:25] "Validated CRI v1 runtime API" Jul 24 12:19:03.975726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.975708 2578 log.go:25] "Validated CRI v1 image API" Jul 24 12:19:03.977144 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.977124 2578 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 24 12:19:03.979150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.979130 2578 fs.go:135] Filesystem UUIDs: map[73d2ade6-0efb-4817-b8e2-56d9603ea862:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e6c39479-2006-4b69-839c-ca6077294ede:/dev/nvme0n1p3] Jul 24 12:19:03.979225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.979150 2578 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 24 12:19:03.983931 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.983818 2578 manager.go:216] Machine: {Timestamp:2026-07-24 12:19:03.982708854 +0000 UTC m=+0.268254549 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3200924 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2784cc32b95feb9c955331c30e01bc SystemUUID:ec2784cc-32b9-5feb-9c95-5331c30e01bc BootID:3a7b45dc-8c3a-41ed-b1af-15577e759e96 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:6b:48:fb:e1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:6b:48:fb:e1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:46:1d:26:9b:db:0b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 24 12:19:03.983931 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.983928 2578 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 24 12:19:03.984032 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.984013 2578 manager.go:232] Version: {KernelVersion:5.14.0-570.129.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260717-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 24 12:19:03.986705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.986671 2578 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 24 12:19:03.986845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.986708 2578 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-132-126.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 24 12:19:03.987363 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.987347 2578 topology_manager.go:138] "Creating topology manager with none policy" Jul 24 12:19:03.987398 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.987365 2578 container_manager_linux.go:309] "Creating device plugin manager" Jul 24 12:19:03.987398 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.987375 2578 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 12:19:03.987941 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.987931 2578 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 24 12:19:03.988725 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.988716 2578 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 24 12:19:03.989141 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.989126 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 12:19:03.991434 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991421 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 24 12:19:03.991588 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991579 2578 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 24 12:19:03.991624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991609 2578 kubelet.go:486] "Attempting to sync node with API server" Jul 24 12:19:03.991624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991619 2578 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 24 12:19:03.991674 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991630 2578 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 24 12:19:03.991674 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991639 2578 kubelet.go:398] "Adding apiserver pod source" Jul 24 12:19:03.991674 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991648 2578 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 24 12:19:03.992013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.991991 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 12:19:03.992079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.992021 2578 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 24 12:19:03.993793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.993777 2578 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 24 12:19:03.995399 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.995383 2578 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 24 12:19:03.995451 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.995435 2578 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 24 12:19:03.998257 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998238 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998265 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998274 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998283 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998296 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998304 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998317 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998326 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 24 12:19:03.998334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998336 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 24 12:19:03.998541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998345 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 24 12:19:03.998541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998368 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 24 12:19:03.998541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.998376 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 24 12:19:03.999046 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.999034 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 24 12:19:03.999077 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:03.999048 2578 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 24 12:19:04.002053 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.002027 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-132-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 24 12:19:04.002122 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.002029 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 24 12:19:04.002122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.002073 2578 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-132-126.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 24 12:19:04.002236 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.002223 2578 server.go:1268] "Started kubelet" Jul 24 12:19:04.002361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.002312 2578 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 24 12:19:04.002444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.002401 2578 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 24 12:19:04.002512 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.002321 2578 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 24 12:19:04.003179 ip-10-0-132-126 systemd[1]: Started Kubernetes Kubelet. Jul 24 12:19:04.004353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.004341 2578 server.go:310] "Adding debug handlers to kubelet server" Jul 24 12:19:04.006186 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.006169 2578 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 24 12:19:04.007308 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.007291 2578 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 24 12:19:04.012115 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012099 2578 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 24 12:19:04.012209 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012104 2578 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 24 12:19:04.012257 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.012227 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.012777 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012752 2578 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 24 12:19:04.012777 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012773 2578 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 24 12:19:04.012926 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012748 2578 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 24 12:19:04.012926 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012916 2578 reconstruct.go:97] "Volume reconstruction finished" Jul 24 12:19:04.012926 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.012926 2578 reconciler.go:29] "Reconciler: start to sync state" Jul 24 12:19:04.013060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013016 2578 factory.go:55] Registering systemd factory Jul 24 12:19:04.013060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013032 2578 factory.go:223] Registration of the systemd container factory successfully Jul 24 12:19:04.013489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013469 2578 factory.go:153] Registering CRI-O factory Jul 24 12:19:04.013489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013487 2578 factory.go:223] Registration of the crio container factory successfully Jul 24 12:19:04.013621 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013567 2578 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 24 12:19:04.013621 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013610 2578 factory.go:103] Registering Raw factory Jul 24 12:19:04.013697 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.013625 2578 manager.go:1196] Started watching for new ooms in manager Jul 24 12:19:04.014952 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.014937 2578 manager.go:349] Starting recovery of all containers Jul 24 12:19:04.016291 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.016270 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c5dsr" Jul 24 12:19:04.019299 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.019270 2578 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-132-126.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 24 12:19:04.019299 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.019288 2578 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 24 12:19:04.020289 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.019421 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-126.ec2.internal.18c5390a5d6d70d3 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-126.ec2.internal,UID:ip-10-0-132-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-132-126.ec2.internal,},FirstTimestamp:2026-07-24 12:19:04.002179283 +0000 UTC m=+0.287724958,LastTimestamp:2026-07-24 12:19:04.002179283 +0000 UTC m=+0.287724958,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-126.ec2.internal,}" Jul 24 12:19:04.024839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.024810 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-c5dsr" Jul 24 12:19:04.027798 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.027615 2578 manager.go:354] Recovery completed Jul 24 12:19:04.032898 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.032884 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.035122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035103 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.035216 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035134 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.035216 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035148 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.035665 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035652 2578 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 24 12:19:04.035665 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035663 2578 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 24 12:19:04.035747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035668 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.035747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.035678 2578 state_mem.go:36] "Initialized new in-memory state store" Jul 24 12:19:04.036394 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.036379 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.036461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.036407 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.036461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.036422 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.036906 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.036840 2578 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-132-126.ec2.internal.18c5390a5f641228 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-132-126.ec2.internal,UID:ip-10-0-132-126.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-132-126.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-132-126.ec2.internal,},FirstTimestamp:2026-07-24 12:19:04.035119656 +0000 UTC m=+0.320665332,LastTimestamp:2026-07-24 12:19:04.035119656 +0000 UTC m=+0.320665332,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-132-126.ec2.internal,}" Jul 24 12:19:04.038342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.038326 2578 policy_none.go:49] "None policy: Start" Jul 24 12:19:04.038342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.038344 2578 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 24 12:19:04.038445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.038356 2578 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 24 12:19:04.040346 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.040335 2578 policy_none.go:47] "Start" Jul 24 12:19:04.080818 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.080797 2578 manager.go:337] "Starting Device Plugin manager" Jul 24 12:19:04.080924 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.080833 2578 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 24 12:19:04.080924 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.080847 2578 server.go:87] "Starting device plugin registration server" Jul 24 12:19:04.081124 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.081113 2578 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 24 12:19:04.081172 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.081126 2578 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 24 12:19:04.081280 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.081257 2578 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 24 12:19:04.081402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.081382 2578 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 24 12:19:04.081402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.081400 2578 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 24 12:19:04.082046 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.082026 2578 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 24 12:19:04.082110 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.082074 2578 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.140475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.140433 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 24 12:19:04.142652 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.141827 2578 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 24 12:19:04.142652 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.141875 2578 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 24 12:19:04.142652 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.141905 2578 kubelet.go:2448] "Starting kubelet main sync loop" Jul 24 12:19:04.142652 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.141948 2578 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 24 12:19:04.144404 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.144384 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:04.181890 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.181856 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.182880 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.182862 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.182946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.182897 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.182946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.182914 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.182946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.182940 2578 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.191557 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.191513 2578 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.191557 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.191536 2578 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-132-126.ec2.internal\": node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.206302 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.206283 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.242205 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.242162 2578 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal"] Jul 24 12:19:04.242282 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.242248 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.244155 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.244137 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.244246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.244167 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.244246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.244182 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.246532 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.246519 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.246659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.246644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.246693 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.246685 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.247855 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247836 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.247983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247867 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.247983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247881 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.247983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247886 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.247983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247914 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.247983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.247926 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.250259 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.250243 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.250351 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.250271 2578 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 24 12:19:04.251583 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.251566 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientMemory" Jul 24 12:19:04.251657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.251597 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasNoDiskPressure" Jul 24 12:19:04.251657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.251608 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeHasSufficientPID" Jul 24 12:19:04.273878 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.273861 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-126.ec2.internal\" not found" node="ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.278345 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.278327 2578 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-132-126.ec2.internal\" not found" node="ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.307144 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.307118 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.314976 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.314947 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.315072 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.314998 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.315072 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.315030 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfcbafe564693f13913bc13ffce62314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-126.ec2.internal\" (UID: \"dfcbafe564693f13913bc13ffce62314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.407292 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.407253 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.415723 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415702 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.415796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.415796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfcbafe564693f13913bc13ffce62314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-126.ec2.internal\" (UID: \"dfcbafe564693f13913bc13ffce62314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.415905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415813 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.415905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415815 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/dfcbafe564693f13913bc13ffce62314-config\") pod \"kube-apiserver-proxy-ip-10-0-132-126.ec2.internal\" (UID: \"dfcbafe564693f13913bc13ffce62314\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.415905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.415827 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b4250fda63cae61110e2071d94ef017-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal\" (UID: \"8b4250fda63cae61110e2071d94ef017\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.508185 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.508105 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.576644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.576610 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.581818 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.581694 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.608901 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.608873 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.709416 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.709379 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.810064 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:04.809982 2578 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-132-126.ec2.internal\" not found" Jul 24 12:19:04.832204 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.832169 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:04.913016 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.912991 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.919970 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.919951 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:04.924691 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.924677 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 12:19:04.925454 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.925442 2578 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" Jul 24 12:19:04.932534 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.932521 2578 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 24 12:19:04.957692 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.957664 2578 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 24 12:19:04.958093 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.957822 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 12:19:04.958093 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.957852 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 12:19:04.958093 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.957848 2578 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 24 12:19:04.991992 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:04.991962 2578 apiserver.go:52] "Watching apiserver" Jul 24 12:19:05.002939 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.002918 2578 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 24 12:19:05.004687 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.004654 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-w7wlm","kube-system/konnectivity-agent-ctvkd","kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc","openshift-dns/node-resolver-4blg2","openshift-image-registry/node-ca-hkz6p","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal","openshift-multus/multus-ssk98","openshift-cluster-node-tuning-operator/tuned-xn5tx","openshift-multus/multus-additional-cni-plugins-wf5j6","openshift-multus/network-metrics-daemon-5ss9t","openshift-network-diagnostics/network-check-target-ln2m5","openshift-network-operator/iptables-alerter-8q88w","openshift-ovn-kubernetes/ovnkube-node-mg9vw"] Jul 24 12:19:05.007322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.007300 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.009256 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009215 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.009394 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009305 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.009394 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009390 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 24 12:19:05.009512 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009445 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bvjfd\"" Jul 24 12:19:05.009563 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009509 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.009609 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009574 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 24 12:19:05.009660 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.009639 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 24 12:19:05.011141 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.011120 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 24 12:19:05.011260 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.011142 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.011260 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.011217 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.011495 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.011482 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-qldps\"" Jul 24 12:19:05.011771 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.011756 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.012240 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.012224 2578 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 24 12:19:05.012987 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.012972 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 24 12:19:05.013273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013259 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.013361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013342 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-zctxn\"" Jul 24 12:19:05.013464 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013446 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 24 12:19:05.013526 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013452 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 24 12:19:05.013585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013532 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.013676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013661 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 24 12:19:05.013898 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.013885 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.015252 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.015237 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.015313 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.015302 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 24 12:19:05.015459 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.015442 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.015595 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.015579 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-ghsb5\"" Jul 24 12:19:05.016759 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.016744 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.018321 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.018307 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.018401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.018364 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-pmrxk\"" Jul 24 12:19:05.018401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.018394 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.018972 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.018957 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.019527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.019649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019541 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-sys-fs\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.019649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-system-cni-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.019649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019587 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4dbc12-4a42-425f-9dde-db151aaab3a9-iptables-alerter-script\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.019649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-node-log\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-script-lib\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-device-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019699 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-var-lib-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019723 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-ovn\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019759 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019787 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019809 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019825 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-kubelet\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019840 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-etc-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019874 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-bin\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.019901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019896 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019936 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4dbc12-4a42-425f-9dde-db151aaab3a9-host-slash\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019962 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-systemd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.019982 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-systemd-units\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020000 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-slash\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020006 2578 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-netns\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020065 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-log-socket\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020086 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovn-node-metrics-cert\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020104 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-socket-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020118 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-registration-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020139 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-os-release\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020163 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lckw\" (UniqueName: \"kubernetes.io/projected/bf4dbc12-4a42-425f-9dde-db151aaab3a9-kube-api-access-7lckw\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-netd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.020377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020206 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-env-overrides\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020230 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cwlm\" (UniqueName: \"kubernetes.io/projected/84600804-b089-4a90-824c-73b40ac14434-kube-api-access-5cwlm\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020247 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-config\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020261 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x4vr\" (UniqueName: \"kubernetes.io/projected/ff4c9d68-31d2-41ab-a01c-305c19e02027-kube-api-access-8x4vr\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020280 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-etc-selinux\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020305 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-cnibin\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020350 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4mpp\" (UniqueName: \"kubernetes.io/projected/4cb9d1e1-a757-4039-850e-91715488a92d-kube-api-access-m4mpp\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020461 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020677 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020703 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-tbqh6\"" Jul 24 12:19:05.021089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.020703 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.021515 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.021257 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.022926 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.022900 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 24 12:19:05.023037 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.023023 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-99bkf\"" Jul 24 12:19:05.023107 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.023092 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 24 12:19:05.023532 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.023517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.025234 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.025218 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 24 12:19:05.025322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.025241 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-ww6pg\"" Jul 24 12:19:05.025703 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.025688 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.027107 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.027092 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 24 12:19:05.027219 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.027179 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 24 12:19:05.027219 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.027218 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-g528b\"" Jul 24 12:19:05.028105 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.028091 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.028170 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.028152 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:05.030792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.030776 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.031097 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.031067 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:05.032623 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.032573 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-23 12:14:04 +0000 UTC" deadline="2028-03-09 20:42:00.126040257 +0000 UTC" Jul 24 12:19:05.032623 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.032623 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14264h22m55.093420912s" Jul 24 12:19:05.034332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.034315 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:05.034394 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.034372 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:05.049397 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.049378 2578 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccn57" Jul 24 12:19:05.056103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.056084 2578 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-ccn57" Jul 24 12:19:05.113657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.113635 2578 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 24 12:19:05.121350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-socket-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-registration-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-netd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121384 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x4vr\" (UniqueName: \"kubernetes.io/projected/ff4c9d68-31d2-41ab-a01c-305c19e02027-kube-api-access-8x4vr\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121405 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121429 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-config\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121454 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-registration-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121455 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-netd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121475 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1086883-e355-4c97-9849-9efa8b112d13-host\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121497 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-multus\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-socket-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-etc-kubernetes\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121556 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/598c1f25-29f8-4ce5-acf0-362bcc9c4854-tmp-dir\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121579 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-etc-selinux\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-cnibin\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121615 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m4mpp\" (UniqueName: \"kubernetes.io/projected/4cb9d1e1-a757-4039-850e-91715488a92d-kube-api-access-m4mpp\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121640 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-os-release\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121658 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-sys\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121662 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-cnibin\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121703 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-etc-selinux\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121765 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4dbc12-4a42-425f-9dde-db151aaab3a9-iptables-alerter-script\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-node-log\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121831 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121841 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-node-log\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-script-lib\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121895 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-bin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121919 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-conf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.121946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121941 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-dbus\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121966 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-device-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.121989 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-var-lib-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122010 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbwzl\" (UniqueName: \"kubernetes.io/projected/e1086883-e355-4c97-9849-9efa8b112d13-kube-api-access-xbwzl\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122039 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df7740b-666b-4ecb-87d4-d20b855117f7-konnectivity-ca\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122039 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-device-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122065 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-var-lib-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122068 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122091 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122162 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-kubelet\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122210 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-kubelet\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122241 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-etc-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-bin\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-etc-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cnibin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122334 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/bf4dbc12-4a42-425f-9dde-db151aaab3a9-iptables-alerter-script\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.122726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122350 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-cni-bin\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122353 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122397 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-script-lib\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122399 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4dbc12-4a42-425f-9dde-db151aaab3a9-host-slash\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122443 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-systemd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/bf4dbc12-4a42-425f-9dde-db151aaab3a9-host-slash\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122470 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-kubelet\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122505 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-systemd\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122509 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-hostroot\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122545 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rq7sg\" (UniqueName: \"kubernetes.io/projected/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-kube-api-access-rq7sg\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysconfig\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122585 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovnkube-config\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122616 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-var-lib-kubelet\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122639 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-netns\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122640 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122669 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-binary-copy\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.123379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122704 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-openvswitch\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122739 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovn-node-metrics-cert\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-system-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122785 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-socket-dir-parent\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/4cb9d1e1-a757-4039-850e-91715488a92d-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122807 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-tmp\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122845 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-os-release\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122868 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7lckw\" (UniqueName: \"kubernetes.io/projected/bf4dbc12-4a42-425f-9dde-db151aaab3a9-kube-api-access-7lckw\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122883 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-env-overrides\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122912 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-run\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122944 2578 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122968 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-os-release\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.122993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-lib-modules\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cwlm\" (UniqueName: \"kubernetes.io/projected/84600804-b089-4a90-824c-73b40ac14434-kube-api-access-5cwlm\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cni-binary-copy\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123085 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-conf-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123108 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123128 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-systemd\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123152 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmgf\" (UniqueName: \"kubernetes.io/projected/598c1f25-29f8-4ce5-acf0-362bcc9c4854-kube-api-access-sqmgf\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123175 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-modprobe-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123226 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-host\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123249 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-tuned\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123276 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ff4c9d68-31d2-41ab-a01c-305c19e02027-env-overrides\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-sys-fs\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-system-cni-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123335 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-sys-fs\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-system-cni-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123374 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/84600804-b089-4a90-824c-73b40ac14434-kubelet-dir\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123414 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-multus-certs\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123440 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df7740b-666b-4ecb-87d4-d20b855117f7-agent-certs\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123463 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-kubelet-config\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-ovn\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123512 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1086883-e355-4c97-9849-9efa8b112d13-serviceca\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.124559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-run-ovn\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123537 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123565 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123609 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq8mf\" (UniqueName: \"kubernetes.io/projected/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-kube-api-access-nq8mf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123635 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123652 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/4cb9d1e1-a757-4039-850e-91715488a92d-tuning-conf-dir\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123667 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-k8s-cni-cncf-io\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123633 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123691 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-daemon-config\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123728 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qthk5\" (UniqueName: \"kubernetes.io/projected/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-kube-api-access-qthk5\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/598c1f25-29f8-4ce5-acf0-362bcc9c4854-hosts-file\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123766 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-systemd-units\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-slash\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123808 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-systemd-units\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123827 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-netns\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123839 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-slash\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123852 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-log-socket\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-host-run-netns\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.125495 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-kubernetes\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.125495 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.123916 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ff4c9d68-31d2-41ab-a01c-305c19e02027-log-socket\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.126430 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.126408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ff4c9d68-31d2-41ab-a01c-305c19e02027-ovn-node-metrics-cert\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.130251 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.130163 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4mpp\" (UniqueName: \"kubernetes.io/projected/4cb9d1e1-a757-4039-850e-91715488a92d-kube-api-access-m4mpp\") pod \"multus-additional-cni-plugins-wf5j6\" (UID: \"4cb9d1e1-a757-4039-850e-91715488a92d\") " pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.130345 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.130281 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x4vr\" (UniqueName: \"kubernetes.io/projected/ff4c9d68-31d2-41ab-a01c-305c19e02027-kube-api-access-8x4vr\") pod \"ovnkube-node-mg9vw\" (UID: \"ff4c9d68-31d2-41ab-a01c-305c19e02027\") " pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.131334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.130997 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lckw\" (UniqueName: \"kubernetes.io/projected/bf4dbc12-4a42-425f-9dde-db151aaab3a9-kube-api-access-7lckw\") pod \"iptables-alerter-8q88w\" (UID: \"bf4dbc12-4a42-425f-9dde-db151aaab3a9\") " pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.131334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.131289 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cwlm\" (UniqueName: \"kubernetes.io/projected/84600804-b089-4a90-824c-73b40ac14434-kube-api-access-5cwlm\") pod \"aws-ebs-csi-driver-node-gtlkc\" (UID: \"84600804-b089-4a90-824c-73b40ac14434\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.216732 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.216701 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b4250fda63cae61110e2071d94ef017.slice/crio-9afb6246228c7f80d7046ccce63db5e87b439ae6f83d483bdcdec70bd13e5c1f WatchSource:0}: Error finding container 9afb6246228c7f80d7046ccce63db5e87b439ae6f83d483bdcdec70bd13e5c1f: Status 404 returned error can't find the container with id 9afb6246228c7f80d7046ccce63db5e87b439ae6f83d483bdcdec70bd13e5c1f Jul 24 12:19:05.217066 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.217046 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddfcbafe564693f13913bc13ffce62314.slice/crio-f5aa4eca7ea04be50008be4e2fa7316706a4c34971979f9049e17a8c62111860 WatchSource:0}: Error finding container f5aa4eca7ea04be50008be4e2fa7316706a4c34971979f9049e17a8c62111860: Status 404 returned error can't find the container with id f5aa4eca7ea04be50008be4e2fa7316706a4c34971979f9049e17a8c62111860 Jul 24 12:19:05.221140 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.221123 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 12:19:05.224889 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.224867 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1086883-e355-4c97-9849-9efa8b112d13-serviceca\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.224979 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.224905 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq8mf\" (UniqueName: \"kubernetes.io/projected/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-kube-api-access-nq8mf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.224979 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.224930 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:05.224979 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.224956 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-k8s-cni-cncf-io\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225101 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225013 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-k8s-cni-cncf-io\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225138 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-daemon-config\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225138 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qthk5\" (UniqueName: \"kubernetes.io/projected/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-kube-api-access-qthk5\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225240 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225226 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/598c1f25-29f8-4ce5-acf0-362bcc9c4854-hosts-file\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.225295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-kubernetes\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.225295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225288 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.225389 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225321 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.225389 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225331 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/598c1f25-29f8-4ce5-acf0-362bcc9c4854-hosts-file\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.225389 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225346 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1086883-e355-4c97-9849-9efa8b112d13-host\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.225389 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225358 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-kubernetes\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.225389 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.225384 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225396 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/e1086883-e355-4c97-9849-9efa8b112d13-host\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/e1086883-e355-4c97-9849-9efa8b112d13-serviceca\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225413 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-multus\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-multus\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.225461 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.225476 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:05.725438012 +0000 UTC m=+2.010983676 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225510 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-etc-kubernetes\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.225530 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:05.725512762 +0000 UTC m=+2.011058428 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-etc-kubernetes\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225562 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/598c1f25-29f8-4ce5-acf0-362bcc9c4854-tmp-dir\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.225598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225596 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-os-release\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225620 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-sys\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225632 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-daemon-config\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225646 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-bin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-cni-bin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225688 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-os-release\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225706 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-conf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225730 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-dbus\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225755 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbwzl\" (UniqueName: \"kubernetes.io/projected/e1086883-e355-4c97-9849-9efa8b112d13-kube-api-access-xbwzl\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-sys\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225773 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df7740b-666b-4ecb-87d4-d20b855117f7-konnectivity-ca\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225797 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225804 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-conf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225812 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cnibin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225840 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-kubelet\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225844 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-dbus\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cnibin\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225852 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/598c1f25-29f8-4ce5-acf0-362bcc9c4854-tmp-dir\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.226092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225879 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-var-lib-kubelet\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-hostroot\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225899 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225913 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-hostroot\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225911 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rq7sg\" (UniqueName: \"kubernetes.io/projected/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-kube-api-access-rq7sg\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysconfig\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225961 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-var-lib-kubelet\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225976 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-netns\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.225992 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-system-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226013 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-socket-dir-parent\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226015 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-var-lib-kubelet\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226021 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysconfig\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226029 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-tmp\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226023 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-netns\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226067 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-run\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226092 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-lib-modules\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226103 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-run\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226115 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cni-binary-copy\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.226896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226067 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-socket-dir-parent\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226068 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-system-cni-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-conf-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226210 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-systemd\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226235 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmgf\" (UniqueName: \"kubernetes.io/projected/598c1f25-29f8-4ce5-acf0-362bcc9c4854-kube-api-access-sqmgf\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-multus-conf-dir\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226258 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-modprobe-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226273 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-systemd\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-host\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226235 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-lib-modules\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226306 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-tuned\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226324 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/1df7740b-666b-4ecb-87d4-d20b855117f7-konnectivity-ca\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226331 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-multus-certs\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226356 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df7740b-666b-4ecb-87d4-d20b855117f7-agent-certs\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226370 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-host-run-multus-certs\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226382 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-kubelet-config\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226389 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-host\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.227955 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226355 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-modprobe-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.228765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226332 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-sysctl-d\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.228765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226467 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/bf78f542-b932-49f4-9476-ed5cccaaaccd-kubelet-config\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.228765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.226592 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-cni-binary-copy\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.228765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.228720 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-tmp\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.228765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.228749 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-etc-tuned\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.228944 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.228928 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/1df7740b-666b-4ecb-87d4-d20b855117f7-agent-certs\") pod \"konnectivity-agent-ctvkd\" (UID: \"1df7740b-666b-4ecb-87d4-d20b855117f7\") " pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.230702 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.230680 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:05.230702 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.230699 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:05.230995 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.230708 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:05.230995 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.230753 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:05.730739007 +0000 UTC m=+2.016284680 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:05.232702 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.232684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq8mf\" (UniqueName: \"kubernetes.io/projected/a67ffa2b-99ba-4114-b357-8e3d42a41f3d-kube-api-access-nq8mf\") pod \"tuned-xn5tx\" (UID: \"a67ffa2b-99ba-4114-b357-8e3d42a41f3d\") " pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.232856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.232813 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbwzl\" (UniqueName: \"kubernetes.io/projected/e1086883-e355-4c97-9849-9efa8b112d13-kube-api-access-xbwzl\") pod \"node-ca-hkz6p\" (UID: \"e1086883-e355-4c97-9849-9efa8b112d13\") " pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.232856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.232816 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qthk5\" (UniqueName: \"kubernetes.io/projected/8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2-kube-api-access-qthk5\") pod \"multus-ssk98\" (UID: \"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2\") " pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.234319 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.234300 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmgf\" (UniqueName: \"kubernetes.io/projected/598c1f25-29f8-4ce5-acf0-362bcc9c4854-kube-api-access-sqmgf\") pod \"node-resolver-4blg2\" (UID: \"598c1f25-29f8-4ce5-acf0-362bcc9c4854\") " pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.234393 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.234357 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rq7sg\" (UniqueName: \"kubernetes.io/projected/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-kube-api-access-rq7sg\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.340443 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.340357 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" Jul 24 12:19:05.351448 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.351420 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4cb9d1e1_a757_4039_850e_91715488a92d.slice/crio-224f7d2804603881fa7893d4755afea1b8c9f6cfa55c06c689523ae2eeb1b5a2 WatchSource:0}: Error finding container 224f7d2804603881fa7893d4755afea1b8c9f6cfa55c06c689523ae2eeb1b5a2: Status 404 returned error can't find the container with id 224f7d2804603881fa7893d4755afea1b8c9f6cfa55c06c689523ae2eeb1b5a2 Jul 24 12:19:05.357727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.357707 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8q88w" Jul 24 12:19:05.366830 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.366806 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf4dbc12_4a42_425f_9dde_db151aaab3a9.slice/crio-6a36f4f05c7fbf5e8bf47f8b4e173f6c8cdbda27188f1ba5f682b6f3622bc09a WatchSource:0}: Error finding container 6a36f4f05c7fbf5e8bf47f8b4e173f6c8cdbda27188f1ba5f682b6f3622bc09a: Status 404 returned error can't find the container with id 6a36f4f05c7fbf5e8bf47f8b4e173f6c8cdbda27188f1ba5f682b6f3622bc09a Jul 24 12:19:05.377282 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.377244 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:05.386014 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.385988 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff4c9d68_31d2_41ab_a01c_305c19e02027.slice/crio-b744c616e06ef451f4c1f7390709bd37b8b0bde9300b1fc1ed8e51a4e14bf6d1 WatchSource:0}: Error finding container b744c616e06ef451f4c1f7390709bd37b8b0bde9300b1fc1ed8e51a4e14bf6d1: Status 404 returned error can't find the container with id b744c616e06ef451f4c1f7390709bd37b8b0bde9300b1fc1ed8e51a4e14bf6d1 Jul 24 12:19:05.390191 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.390175 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" Jul 24 12:19:05.398926 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.398899 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84600804_b089_4a90_824c_73b40ac14434.slice/crio-9cf82a5e729d46cd37b017b8d733c7e03692486fcb65719d483bbd35950e3700 WatchSource:0}: Error finding container 9cf82a5e729d46cd37b017b8d733c7e03692486fcb65719d483bbd35950e3700: Status 404 returned error can't find the container with id 9cf82a5e729d46cd37b017b8d733c7e03692486fcb65719d483bbd35950e3700 Jul 24 12:19:05.406420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.406397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4blg2" Jul 24 12:19:05.416562 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.416530 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod598c1f25_29f8_4ce5_acf0_362bcc9c4854.slice/crio-1c6bdcb049f36676fb2d3c3f7981805469c478c97fbc5df1dcaf5ecd47c96b60 WatchSource:0}: Error finding container 1c6bdcb049f36676fb2d3c3f7981805469c478c97fbc5df1dcaf5ecd47c96b60: Status 404 returned error can't find the container with id 1c6bdcb049f36676fb2d3c3f7981805469c478c97fbc5df1dcaf5ecd47c96b60 Jul 24 12:19:05.426695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.426673 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-hkz6p" Jul 24 12:19:05.435362 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.435338 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1086883_e355_4c97_9849_9efa8b112d13.slice/crio-55c1f82c006ba50360adede6ad44110de06933d3d2cf0a70ba3bd8f5eea0f114 WatchSource:0}: Error finding container 55c1f82c006ba50360adede6ad44110de06933d3d2cf0a70ba3bd8f5eea0f114: Status 404 returned error can't find the container with id 55c1f82c006ba50360adede6ad44110de06933d3d2cf0a70ba3bd8f5eea0f114 Jul 24 12:19:05.453070 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.453033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" Jul 24 12:19:05.461588 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.461559 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda67ffa2b_99ba_4114_b357_8e3d42a41f3d.slice/crio-2cb32ad8c00ce6e3638814afe440f73488eea3dfea08c96a36e488392d3f03db WatchSource:0}: Error finding container 2cb32ad8c00ce6e3638814afe440f73488eea3dfea08c96a36e488392d3f03db: Status 404 returned error can't find the container with id 2cb32ad8c00ce6e3638814afe440f73488eea3dfea08c96a36e488392d3f03db Jul 24 12:19:05.469846 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.469822 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ssk98" Jul 24 12:19:05.474021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.473998 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:05.478722 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.478693 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b403d5c_4ef1_4d8e_8d34_7b4e1f762de2.slice/crio-d17c3cbfc3b2a63b84fa875dc2e455e5a6b87f9165543d3ee8a7567f4c744854 WatchSource:0}: Error finding container d17c3cbfc3b2a63b84fa875dc2e455e5a6b87f9165543d3ee8a7567f4c744854: Status 404 returned error can't find the container with id d17c3cbfc3b2a63b84fa875dc2e455e5a6b87f9165543d3ee8a7567f4c744854 Jul 24 12:19:05.483565 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:05.483542 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1df7740b_666b_4ecb_87d4_d20b855117f7.slice/crio-d0a0633f5cb0acd591595838b75d2a5e94661ef1dfd11c1feed0ee67edd4f5c9 WatchSource:0}: Error finding container d0a0633f5cb0acd591595838b75d2a5e94661ef1dfd11c1feed0ee67edd4f5c9: Status 404 returned error can't find the container with id d0a0633f5cb0acd591595838b75d2a5e94661ef1dfd11c1feed0ee67edd4f5c9 Jul 24 12:19:05.535364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.535336 2578 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:05.731341 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.731252 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:05.731341 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.731312 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:05.731340 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731461 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731461 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731491 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731493 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731504 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:06.731506261 +0000 UTC m=+3.017051926 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:05.731561 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731556 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:06.73153591 +0000 UTC m=+3.017081589 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:05.731842 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:05.731576 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:06.731565814 +0000 UTC m=+3.017111485 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:06.017780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.017692 2578 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:06.059712 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.058905 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 12:14:05 +0000 UTC" deadline="2028-03-14 18:06:08.688772656 +0000 UTC" Jul 24 12:19:06.059712 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.058947 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14381h47m2.629830856s" Jul 24 12:19:06.157053 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.156985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" event={"ID":"dfcbafe564693f13913bc13ffce62314","Type":"ContainerStarted","Data":"f5aa4eca7ea04be50008be4e2fa7316706a4c34971979f9049e17a8c62111860"} Jul 24 12:19:06.172914 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.172871 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" event={"ID":"8b4250fda63cae61110e2071d94ef017","Type":"ContainerStarted","Data":"9afb6246228c7f80d7046ccce63db5e87b439ae6f83d483bdcdec70bd13e5c1f"} Jul 24 12:19:06.178568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.178439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ctvkd" event={"ID":"1df7740b-666b-4ecb-87d4-d20b855117f7","Type":"ContainerStarted","Data":"d0a0633f5cb0acd591595838b75d2a5e94661ef1dfd11c1feed0ee67edd4f5c9"} Jul 24 12:19:06.182427 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.182395 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ssk98" event={"ID":"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2","Type":"ContainerStarted","Data":"d17c3cbfc3b2a63b84fa875dc2e455e5a6b87f9165543d3ee8a7567f4c744854"} Jul 24 12:19:06.191377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.191312 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" event={"ID":"a67ffa2b-99ba-4114-b357-8e3d42a41f3d","Type":"ContainerStarted","Data":"2cb32ad8c00ce6e3638814afe440f73488eea3dfea08c96a36e488392d3f03db"} Jul 24 12:19:06.200442 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.200404 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" event={"ID":"84600804-b089-4a90-824c-73b40ac14434","Type":"ContainerStarted","Data":"9cf82a5e729d46cd37b017b8d733c7e03692486fcb65719d483bbd35950e3700"} Jul 24 12:19:06.202688 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.202652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"b744c616e06ef451f4c1f7390709bd37b8b0bde9300b1fc1ed8e51a4e14bf6d1"} Jul 24 12:19:06.205921 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.205892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkz6p" event={"ID":"e1086883-e355-4c97-9849-9efa8b112d13","Type":"ContainerStarted","Data":"55c1f82c006ba50360adede6ad44110de06933d3d2cf0a70ba3bd8f5eea0f114"} Jul 24 12:19:06.208965 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.208935 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4blg2" event={"ID":"598c1f25-29f8-4ce5-acf0-362bcc9c4854","Type":"ContainerStarted","Data":"1c6bdcb049f36676fb2d3c3f7981805469c478c97fbc5df1dcaf5ecd47c96b60"} Jul 24 12:19:06.215057 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.215027 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8q88w" event={"ID":"bf4dbc12-4a42-425f-9dde-db151aaab3a9","Type":"ContainerStarted","Data":"6a36f4f05c7fbf5e8bf47f8b4e173f6c8cdbda27188f1ba5f682b6f3622bc09a"} Jul 24 12:19:06.221910 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.221865 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerStarted","Data":"224f7d2804603881fa7893d4755afea1b8c9f6cfa55c06c689523ae2eeb1b5a2"} Jul 24 12:19:06.740009 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.739908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:06.740009 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.739967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:06.740241 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.740042 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:06.740329 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740232 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:06.740329 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740288 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:06.740329 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740302 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:06.740493 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740374 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:08.740351576 +0000 UTC m=+5.025897243 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:06.740857 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740800 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:06.740933 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740858 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:08.740843304 +0000 UTC m=+5.026388974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:06.740933 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740918 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:06.741049 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:06.740948 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:08.74093742 +0000 UTC m=+5.026483104 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:06.963191 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:06.960569 2578 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:07.059372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.059182 2578 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-23 12:14:05 +0000 UTC" deadline="2028-04-30 14:30:45.169185804 +0000 UTC" Jul 24 12:19:07.059372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.059264 2578 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15506h11m38.109927142s" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.144723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.144852 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:07.144871 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.144884 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:07.144959 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:07.145350 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:07.145031 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:07.342601 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:07.342490 2578 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 24 12:19:08.759742 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:08.759689 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:08.760175 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:08.759772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:08.760175 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:08.759807 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:08.760175 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.759959 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:08.760175 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760026 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:12.760005822 +0000 UTC m=+9.045551493 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:08.760500 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760476 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:08.760587 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760514 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:08.760587 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760530 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:08.760698 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760584 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:12.760566399 +0000 UTC m=+9.046112070 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:08.760698 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760636 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:08.760698 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:08.760660 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:12.760651658 +0000 UTC m=+9.046197323 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:09.142748 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:09.142661 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:09.142924 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:09.142811 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:09.143242 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:09.143216 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:09.143382 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:09.143361 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:09.143437 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:09.143424 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:09.143523 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:09.143503 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:11.142888 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:11.142848 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:11.143430 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:11.142993 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:11.143510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:11.143427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:11.143571 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:11.143543 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:11.143626 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:11.143590 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:11.143678 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:11.143659 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:12.798366 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:12.798318 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:12.798396 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:12.798426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798506 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798532 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798542 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798559 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798579 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798613 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:20.798590538 +0000 UTC m=+17.084136228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798634 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:20.798626449 +0000 UTC m=+17.084172115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:12.798903 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:12.798646 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:20.798640155 +0000 UTC m=+17.084185820 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:13.143272 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:13.143048 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:13.143272 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:13.143156 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:13.143769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:13.143562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:13.143769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:13.143584 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:13.143769 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:13.143675 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:13.143769 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:13.143739 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:15.143006 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:15.142975 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:15.143453 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:15.143073 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:15.143453 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:15.143360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:15.143453 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:15.143426 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:15.143582 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:15.143458 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:15.143582 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:15.143500 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:17.142821 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:17.142781 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:17.143223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:17.142798 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:17.143223 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:17.142897 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:17.143223 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:17.142970 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:17.143223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:17.142804 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:17.143223 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:17.143058 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:19.142387 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:19.142352 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:19.142857 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:19.142391 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:19.142857 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:19.142397 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:19.142857 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:19.142489 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:19.142857 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:19.142624 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:19.142857 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:19.142724 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:20.854207 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:20.854152 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:20.854207 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:20.854211 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:20.854240 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854352 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854382 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854392 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854356 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854356 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854447 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:36.854429145 +0000 UTC m=+33.139974823 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854518 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:19:36.854504731 +0000 UTC m=+33.140050396 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:20.854745 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:20.854533 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:19:36.854523883 +0000 UTC m=+33.140069548 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:21.142803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:21.142723 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:21.142803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:21.142736 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:21.143016 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:21.142828 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:21.143016 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:21.142848 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:21.143016 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:21.142952 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:21.143137 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:21.143038 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:23.142441 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:23.142400 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:23.142874 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:23.142519 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:23.142925 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:23.142880 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:23.142986 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:23.142965 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:23.143043 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:23.143033 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:23.143125 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:23.143107 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:25.142520 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:25.142471 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:25.142986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:25.142471 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:25.142986 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:25.142612 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:25.142986 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:25.142681 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:25.142986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:25.142487 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:25.142986 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:25.142758 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:26.285839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:26.285795 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ssk98" event={"ID":"8b403d5c-4ef1-4d8e-8d34-7b4e1f762de2","Type":"ContainerStarted","Data":"7a32f5df1c1a40b1720b1fe34bdd7cd14f235bd7151b0e2a7a4e63a49d73282f"} Jul 24 12:19:26.291765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:26.291730 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" event={"ID":"a67ffa2b-99ba-4114-b357-8e3d42a41f3d","Type":"ContainerStarted","Data":"bdb5a314cfbaa7df0be9c31bd4c0530af67d71bba111b11b4c09a18b52056fa1"} Jul 24 12:19:26.296150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:26.296120 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" event={"ID":"dfcbafe564693f13913bc13ffce62314","Type":"ContainerStarted","Data":"cef98b1c3aa362d0d729b3abc8089f19991665ae4a427dc0d8a267c8118b16cd"} Jul 24 12:19:26.307307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:26.307248 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ssk98" podStartSLOduration=1.849466827 podStartE2EDuration="22.307234183s" totalImagesPullingTime="20.457767356s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:26.306658602 +0000 UTC m=+22.592204287" watchObservedRunningTime="2026-07-24 12:19:26.307234183 +0000 UTC m=+22.592779866" Jul 24 12:19:26.325264 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:26.325207 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-132-126.ec2.internal" podStartSLOduration=21.325166165 podStartE2EDuration="21.325166165s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:26.3251055 +0000 UTC m=+22.610651207" watchObservedRunningTime="2026-07-24 12:19:26.325166165 +0000 UTC m=+22.610711911" Jul 24 12:19:27.142736 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.142409 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:27.142929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.142420 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:27.142929 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:27.142811 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:27.142929 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:27.142917 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:27.143047 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.142467 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:27.143047 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:27.142990 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:27.300550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.300514 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="8dfbee359fa1e1d5672d4c5550e441424e0c88be5274f69d85c5dd7ab8231b03" exitCode=0 Jul 24 12:19:27.301328 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.300617 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"8dfbee359fa1e1d5672d4c5550e441424e0c88be5274f69d85c5dd7ab8231b03"} Jul 24 12:19:27.302857 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.302833 2578 generic.go:356] "Generic (PLEG): container finished" podID="8b4250fda63cae61110e2071d94ef017" containerID="b8f0113f13800ca294b74138bdbbe8cc47be6dfedfb48763e8f9483ea6ae6618" exitCode=0 Jul 24 12:19:27.302961 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.302914 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" event={"ID":"8b4250fda63cae61110e2071d94ef017","Type":"ContainerDied","Data":"b8f0113f13800ca294b74138bdbbe8cc47be6dfedfb48763e8f9483ea6ae6618"} Jul 24 12:19:27.304467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.304441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-ctvkd" event={"ID":"1df7740b-666b-4ecb-87d4-d20b855117f7","Type":"ContainerStarted","Data":"fc8db940ea4a04720fdafc7a170c333823a12a4cc27018f7c8a9c241b959af0f"} Jul 24 12:19:27.305811 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.305787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" event={"ID":"84600804-b089-4a90-824c-73b40ac14434","Type":"ContainerStarted","Data":"b66effbce916efc72a56e54763d846030ead50632b2fe6c006569d3081184a3c"} Jul 24 12:19:27.309246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309222 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"6b25fee1b0d4e8a240c2833282151d26ea0d52f3b34a36bfacd7ab09890b4c9a"} Jul 24 12:19:27.309335 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"ebd69e3e38b04c7ac8e1721049de7dd2f3e4fce231b917752c00ee97d3146a9e"} Jul 24 12:19:27.309406 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"47607c659724db05bc3107d346e71b631bb3a27063e7c1b432bc797aa69e25a4"} Jul 24 12:19:27.309406 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309394 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"f9e3e09fb90b1b4068f215001c1f843e91f387c0dcf1725f5c3bdce101c1de84"} Jul 24 12:19:27.309502 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309439 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"bd42b8b251f224faf1b3ab2547f70b8264301892f654c07f6352360b6343f15c"} Jul 24 12:19:27.309502 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.309483 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"2d21b2a02fb01bcfef40eeb98b9eed57753d24e8e642335ddc2dd3241c2cd7fe"} Jul 24 12:19:27.310774 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.310751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-hkz6p" event={"ID":"e1086883-e355-4c97-9849-9efa8b112d13","Type":"ContainerStarted","Data":"beaba063f973d1f3e9aa69443f560ffbbceb2f4acc01dcebd14c79b32572c94c"} Jul 24 12:19:27.312487 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.312444 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4blg2" event={"ID":"598c1f25-29f8-4ce5-acf0-362bcc9c4854","Type":"ContainerStarted","Data":"a8fd1c65ae3a14b9cba7009bb72a84f894cc088855ac214e7673ac90a178af62"} Jul 24 12:19:27.326613 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.326567 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xn5tx" podStartSLOduration=2.869723853 podStartE2EDuration="23.326548345s" totalImagesPullingTime="20.456824492s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:26.34475065 +0000 UTC m=+22.630296335" watchObservedRunningTime="2026-07-24 12:19:27.326548345 +0000 UTC m=+23.612094029" Jul 24 12:19:27.341022 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.340975 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-4blg2" podStartSLOduration=7.333128958 podStartE2EDuration="23.340959964s" totalImagesPullingTime="16.007831006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:27.340322798 +0000 UTC m=+23.625868484" watchObservedRunningTime="2026-07-24 12:19:27.340959964 +0000 UTC m=+23.626505648" Jul 24 12:19:27.354527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.354485 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-ctvkd" podStartSLOduration=7.413865015 podStartE2EDuration="23.354475059s" totalImagesPullingTime="15.940610044s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:27.354403051 +0000 UTC m=+23.639948735" watchObservedRunningTime="2026-07-24 12:19:27.354475059 +0000 UTC m=+23.640020743" Jul 24 12:19:27.369163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.369117 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-hkz6p" podStartSLOduration=2.887739462 podStartE2EDuration="23.369103701s" totalImagesPullingTime="20.481364239s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:27.368673355 +0000 UTC m=+23.654219040" watchObservedRunningTime="2026-07-24 12:19:27.369103701 +0000 UTC m=+23.654649414" Jul 24 12:19:27.730803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:27.730778 2578 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 24 12:19:28.094210 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.094078 2578 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-24T12:19:27.730798735Z","UUID":"6848f492-b69c-4d3c-ab9a-84100ebb9da5","Handler":null,"Name":"","Endpoint":""} Jul 24 12:19:28.096270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.096242 2578 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 24 12:19:28.096418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.096286 2578 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 24 12:19:28.317898 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.317862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" event={"ID":"8b4250fda63cae61110e2071d94ef017","Type":"ContainerStarted","Data":"91aea9f8027927c07a9275a91fd2731d340d5ab37be5237c370da558120c0ca6"} Jul 24 12:19:28.320279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.320247 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" event={"ID":"84600804-b089-4a90-824c-73b40ac14434","Type":"ContainerStarted","Data":"8cd0ae27445d3daa872b667785057544eaf6d17c228d8937a8f49c21af20e54c"} Jul 24 12:19:28.321868 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.321843 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8q88w" event={"ID":"bf4dbc12-4a42-425f-9dde-db151aaab3a9","Type":"ContainerStarted","Data":"a7253fbdb36bd0b2654fad06d3f89df41f864239444effdaa9e31033197ace47"} Jul 24 12:19:28.347583 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.347483 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8q88w" podStartSLOduration=3.8211958839999998 podStartE2EDuration="24.347462006s" totalImagesPullingTime="20.526266122s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:28.34721224 +0000 UTC m=+24.632757937" watchObservedRunningTime="2026-07-24 12:19:28.347462006 +0000 UTC m=+24.633007692" Jul 24 12:19:28.348788 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:28.348045 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-132-126.ec2.internal" podStartSLOduration=23.348038558 podStartE2EDuration="23.348038558s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:28.33336088 +0000 UTC m=+24.618906583" watchObservedRunningTime="2026-07-24 12:19:28.348038558 +0000 UTC m=+24.633584242" Jul 24 12:19:29.142714 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.142636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:29.142913 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.142743 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:29.142913 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.142851 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:29.142913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:29.142870 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:29.143068 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:29.142955 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:29.143116 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:29.143062 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:29.326870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.326829 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" event={"ID":"84600804-b089-4a90-824c-73b40ac14434","Type":"ContainerStarted","Data":"eb662e50b4ab78914f40e9e3a8eeabc810b43d0dc91ee92f3f0a99a1bec94992"} Jul 24 12:19:29.330565 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.330441 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"d20392d09b9737da9805cd48220a8f4154cd08a1d9e545b416e40da3df2f5add"} Jul 24 12:19:29.344949 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:29.344899 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-gtlkc" podStartSLOduration=2.5173283829999997 podStartE2EDuration="25.344881801s" totalImagesPullingTime="22.827553418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:29.344626099 +0000 UTC m=+25.630171783" watchObservedRunningTime="2026-07-24 12:19:29.344881801 +0000 UTC m=+25.630427485" Jul 24 12:19:30.925345 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:30.925309 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:30.925969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:30.925949 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:31.142139 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:31.142112 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:31.142294 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:31.142250 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:31.142294 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:31.142255 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:31.142294 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:31.142282 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:31.142420 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:31.142370 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:31.142459 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:31.142428 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:31.334936 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:31.334907 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:31.335376 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:31.335357 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-ctvkd" Jul 24 12:19:32.338295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:32.338256 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="ccee22efeca241085a1b9f98623b3790b4eff39cd9ee4fb21a555d065696328d" exitCode=0 Jul 24 12:19:32.339254 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:32.338348 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"ccee22efeca241085a1b9f98623b3790b4eff39cd9ee4fb21a555d065696328d"} Jul 24 12:19:32.342594 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:32.342387 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" event={"ID":"ff4c9d68-31d2-41ab-a01c-305c19e02027","Type":"ContainerStarted","Data":"05eb2df67aac74aee3a53a40dd726f128397d08ff41daeb08c1ed44ddbfb8b37"} Jul 24 12:19:32.384791 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:32.384668 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" podStartSLOduration=7.60020246 podStartE2EDuration="28.384651498s" totalImagesPullingTime="20.784449038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:32.383907946 +0000 UTC m=+28.669453630" watchObservedRunningTime="2026-07-24 12:19:32.384651498 +0000 UTC m=+28.670197183" Jul 24 12:19:33.142704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.142662 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:33.142984 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.142787 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:33.143292 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.143269 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:33.143419 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.143368 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:33.143491 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.143427 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:33.143566 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.143493 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:33.345451 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.345421 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:33.345912 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.345498 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:33.345912 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.345539 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:33.361618 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.361590 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:33.361817 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.361805 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:19:33.397010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.396907 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5ss9t"] Jul 24 12:19:33.397215 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.397036 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:33.397215 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.397155 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:33.397571 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.397551 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ln2m5"] Jul 24 12:19:33.397659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.397636 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:33.397749 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.397730 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:33.399301 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.399279 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w7wlm"] Jul 24 12:19:33.399425 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:33.399359 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:33.399486 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:33.399460 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:34.348906 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:34.348867 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="c4808ca91ec89fef375d31f679ea0cc2bfbb9b6c04b907f68a31ac8d5bb5290f" exitCode=0 Jul 24 12:19:34.349304 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:34.348952 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"c4808ca91ec89fef375d31f679ea0cc2bfbb9b6c04b907f68a31ac8d5bb5290f"} Jul 24 12:19:35.143007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:35.142967 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:35.143206 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:35.143012 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:35.143206 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:35.142969 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:35.143206 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:35.143097 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:35.143372 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:35.143214 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:35.143372 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:35.143332 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:36.358717 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:36.358534 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="28e2d337945dcde3b4e3f3ec795d3d8728600f7ddf1889107ec20d761944bfac" exitCode=0 Jul 24 12:19:36.359110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:36.358599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"28e2d337945dcde3b4e3f3ec795d3d8728600f7ddf1889107ec20d761944bfac"} Jul 24 12:19:36.871512 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:36.871470 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:36.871683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:36.871531 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:36.871683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:36.871560 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:36.871683 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871662 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 24 12:19:36.871683 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871680 2578 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871693 2578 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871707 2578 projected.go:196] Error preparing data for projected volume kube-api-access-wq2p5 for pod openshift-network-diagnostics/network-check-target-ln2m5: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871678 2578 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871747 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret podName:bf78f542-b932-49f4-9476-ed5cccaaaccd nodeName:}" failed. No retries permitted until 2026-07-24 12:20:08.871725822 +0000 UTC m=+65.157271490 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret") pod "global-pull-secret-syncer-w7wlm" (UID: "bf78f542-b932-49f4-9476-ed5cccaaaccd") : object "kube-system"/"original-pull-secret" not registered Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871793 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5 podName:8ec8dd0a-5096-44cc-84d1-b126e867dae2 nodeName:}" failed. No retries permitted until 2026-07-24 12:20:08.871776075 +0000 UTC m=+65.157321739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-wq2p5" (UniqueName: "kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5") pod "network-check-target-ln2m5" (UID: "8ec8dd0a-5096-44cc-84d1-b126e867dae2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 24 12:19:36.871913 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:36.871816 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs podName:7dc1ac0f-2573-444a-ad36-c0d4be8b025a nodeName:}" failed. No retries permitted until 2026-07-24 12:20:08.871805625 +0000 UTC m=+65.157351291 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs") pod "network-metrics-daemon-5ss9t" (UID: "7dc1ac0f-2573-444a-ad36-c0d4be8b025a") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 24 12:19:37.142370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:37.142280 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:37.142370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:37.142340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:37.142566 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:37.142283 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:37.142566 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:37.142420 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-5ss9t" podUID="7dc1ac0f-2573-444a-ad36-c0d4be8b025a" Jul 24 12:19:37.142566 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:37.142486 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ln2m5" podUID="8ec8dd0a-5096-44cc-84d1-b126e867dae2" Jul 24 12:19:37.142566 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:37.142560 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w7wlm" podUID="bf78f542-b932-49f4-9476-ed5cccaaaccd" Jul 24 12:19:39.018654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.018581 2578 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-132-126.ec2.internal" event="NodeReady" Jul 24 12:19:39.019227 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.018789 2578 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 24 12:19:39.054821 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.054787 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:19:39.057213 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.057169 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.060554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.060523 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 24 12:19:39.060663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.060560 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 24 12:19:39.061437 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.061415 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-xlzt9\"" Jul 24 12:19:39.061666 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.061642 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 24 12:19:39.066919 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.066897 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 24 12:19:39.070558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.070411 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-2g8z9"] Jul 24 12:19:39.073290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.073260 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-jslk5"] Jul 24 12:19:39.073411 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.073392 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.074797 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.074778 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.075027 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.075006 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 24 12:19:39.075352 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.075335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 24 12:19:39.075496 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.075476 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-68vm8\"" Jul 24 12:19:39.077933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.077906 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 24 12:19:39.078030 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.077948 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-kcvkr\"" Jul 24 12:19:39.078135 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.078119 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 24 12:19:39.078226 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.078157 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 24 12:19:39.081428 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.081406 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:19:39.085861 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.085840 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jslk5"] Jul 24 12:19:39.087591 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.087567 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2g8z9"] Jul 24 12:19:39.143116 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.143084 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:19:39.143336 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.143229 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:19:39.143409 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.143238 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:19:39.145150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145131 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 12:19:39.145416 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145404 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 12:19:39.145606 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145577 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zhxrp\"" Jul 24 12:19:39.145677 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145644 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 12:19:39.145737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145671 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k4xp5\"" Jul 24 12:19:39.145737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.145728 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 12:19:39.188459 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-cert\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.188592 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188461 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s689q\" (UniqueName: \"kubernetes.io/projected/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-kube-api-access-s689q\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.188592 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188487 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188592 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188592 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188579 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188612 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac784d3-ac39-4697-8db8-90e5b096374f-tmp-dir\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188648 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jr4xm\" (UniqueName: \"kubernetes.io/projected/7ac784d3-ac39-4697-8db8-90e5b096374f-kube-api-access-jr4xm\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188721 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188755 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188796 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-htdph\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.188986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.188869 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac784d3-ac39-4697-8db8-90e5b096374f-config-volume\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.290307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290265 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290328 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290354 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-htdph\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290412 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac784d3-ac39-4697-8db8-90e5b096374f-config-volume\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.290438 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 12:19:39.290514 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.290461 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b56447786-85wbl: secret "image-registry-tls" not found Jul 24 12:19:39.290804 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.290523 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls podName:c58a1211-6ad4-4b3a-a037-de7bef79913b nodeName:}" failed. No retries permitted until 2026-07-24 12:19:39.79050064 +0000 UTC m=+36.076046304 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls") pod "image-registry-5b56447786-85wbl" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b") : secret "image-registry-tls" not found Jul 24 12:19:39.290804 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-cert\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.290804 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290746 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s689q\" (UniqueName: \"kubernetes.io/projected/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-kube-api-access-s689q\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.290804 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290784 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290991 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290815 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.290991 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290869 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.290991 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290898 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac784d3-ac39-4697-8db8-90e5b096374f-tmp-dir\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.290991 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.290929 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.291163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291004 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.291163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291030 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jr4xm\" (UniqueName: \"kubernetes.io/projected/7ac784d3-ac39-4697-8db8-90e5b096374f-kube-api-access-jr4xm\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.291163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291149 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/7ac784d3-ac39-4697-8db8-90e5b096374f-config-volume\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.291402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291378 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.291556 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.291469 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 12:19:39.291556 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.291525 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls podName:7ac784d3-ac39-4697-8db8-90e5b096374f nodeName:}" failed. No retries permitted until 2026-07-24 12:19:39.791508614 +0000 UTC m=+36.077054278 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls") pod "dns-default-2g8z9" (UID: "7ac784d3-ac39-4697-8db8-90e5b096374f") : secret "dns-default-metrics-tls" not found Jul 24 12:19:39.291773 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291751 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7ac784d3-ac39-4697-8db8-90e5b096374f-tmp-dir\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.291950 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.291899 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.292219 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.292178 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.295288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.295268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.295372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.295268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.295432 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.295367 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-cert\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.322043 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.321976 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.322209 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.322175 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jr4xm\" (UniqueName: \"kubernetes.io/projected/7ac784d3-ac39-4697-8db8-90e5b096374f-kube-api-access-jr4xm\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.322407 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.322383 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-htdph\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.322724 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.322679 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s689q\" (UniqueName: \"kubernetes.io/projected/cecbdba0-1239-44c4-ba86-cb670f0e7f6d-kube-api-access-s689q\") pod \"ingress-canary-jslk5\" (UID: \"cecbdba0-1239-44c4-ba86-cb670f0e7f6d\") " pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.350938 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.350718 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd"] Jul 24 12:19:39.353726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.353692 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" Jul 24 12:19:39.357487 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.357464 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 24 12:19:39.357648 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.357506 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gm9gv\"" Jul 24 12:19:39.357648 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.357587 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 24 12:19:39.365514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.365488 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd"] Jul 24 12:19:39.391901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.391868 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-jslk5" Jul 24 12:19:39.492886 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.492854 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rjzfp\" (UniqueName: \"kubernetes.io/projected/2647aee1-a03d-448d-9597-401ac14cf241-kube-api-access-rjzfp\") pod \"migrator-6865c9967-j84gd\" (UID: \"2647aee1-a03d-448d-9597-401ac14cf241\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" Jul 24 12:19:39.593812 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.593724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rjzfp\" (UniqueName: \"kubernetes.io/projected/2647aee1-a03d-448d-9597-401ac14cf241-kube-api-access-rjzfp\") pod \"migrator-6865c9967-j84gd\" (UID: \"2647aee1-a03d-448d-9597-401ac14cf241\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" Jul 24 12:19:39.603143 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.603113 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rjzfp\" (UniqueName: \"kubernetes.io/projected/2647aee1-a03d-448d-9597-401ac14cf241-kube-api-access-rjzfp\") pod \"migrator-6865c9967-j84gd\" (UID: \"2647aee1-a03d-448d-9597-401ac14cf241\") " pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" Jul 24 12:19:39.664711 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.664670 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" Jul 24 12:19:39.795376 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.795337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:39.795525 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.795447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:39.795525 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.795503 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 12:19:39.795525 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.795523 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b56447786-85wbl: secret "image-registry-tls" not found Jul 24 12:19:39.795655 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.795553 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 12:19:39.795655 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.795577 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls podName:c58a1211-6ad4-4b3a-a037-de7bef79913b nodeName:}" failed. No retries permitted until 2026-07-24 12:19:40.795562524 +0000 UTC m=+37.081108187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls") pod "image-registry-5b56447786-85wbl" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b") : secret "image-registry-tls" not found Jul 24 12:19:39.795655 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:39.795608 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls podName:7ac784d3-ac39-4697-8db8-90e5b096374f nodeName:}" failed. No retries permitted until 2026-07-24 12:19:40.795600473 +0000 UTC m=+37.081146136 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls") pod "dns-default-2g8z9" (UID: "7ac784d3-ac39-4697-8db8-90e5b096374f") : secret "dns-default-metrics-tls" not found Jul 24 12:19:39.887968 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.887893 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-zvrkq"] Jul 24 12:19:39.890173 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.890142 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:39.892295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.892273 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 24 12:19:39.893015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.892856 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 24 12:19:39.893015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.892890 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 24 12:19:39.893015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.892932 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 24 12:19:39.893384 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.893366 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sd4k9\"" Jul 24 12:19:39.906341 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.906308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvrkq"] Jul 24 12:19:39.996638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.996599 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11bf7252-0597-488c-a6f0-4711e2e20ae0-crio-socket\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:39.996823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.996656 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:39.996823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.996756 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:39.996917 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.996870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11bf7252-0597-488c-a6f0-4711e2e20ae0-data-volume\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:39.996917 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:39.996894 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zw9hj\" (UniqueName: \"kubernetes.io/projected/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-api-access-zw9hj\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.097318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097283 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11bf7252-0597-488c-a6f0-4711e2e20ae0-crio-socket\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.097318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097324 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097358 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097423 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11bf7252-0597-488c-a6f0-4711e2e20ae0-data-volume\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097444 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zw9hj\" (UniqueName: \"kubernetes.io/projected/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-api-access-zw9hj\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.097477 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.097510 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.097573 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/11bf7252-0597-488c-a6f0-4711e2e20ae0-crio-socket\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.097551 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:40.597530568 +0000 UTC m=+36.883076234 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:40.098088 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.097868 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:40.597848443 +0000 UTC m=+36.883394126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : configmap "kube-rbac-proxy" not found Jul 24 12:19:40.098519 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.098104 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/11bf7252-0597-488c-a6f0-4711e2e20ae0-data-volume\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.107265 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.107214 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zw9hj\" (UniqueName: \"kubernetes.io/projected/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-api-access-zw9hj\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.299416 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.299236 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd"] Jul 24 12:19:40.302167 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.302141 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-jslk5"] Jul 24 12:19:40.306860 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:40.305986 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2647aee1_a03d_448d_9597_401ac14cf241.slice/crio-abe1c8f43087c1f8734a932ae1d3edbfbc4d457c202a596896d5fabd2b67b5a3 WatchSource:0}: Error finding container abe1c8f43087c1f8734a932ae1d3edbfbc4d457c202a596896d5fabd2b67b5a3: Status 404 returned error can't find the container with id abe1c8f43087c1f8734a932ae1d3edbfbc4d457c202a596896d5fabd2b67b5a3 Jul 24 12:19:40.311048 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:40.310936 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcecbdba0_1239_44c4_ba86_cb670f0e7f6d.slice/crio-347e57c17e673e4ac5d5235a5e6017f2266c4290c81912f9695ae7e22d3c0598 WatchSource:0}: Error finding container 347e57c17e673e4ac5d5235a5e6017f2266c4290c81912f9695ae7e22d3c0598: Status 404 returned error can't find the container with id 347e57c17e673e4ac5d5235a5e6017f2266c4290c81912f9695ae7e22d3c0598 Jul 24 12:19:40.372011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.371974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerStarted","Data":"3bfefa9c23455e6c46f71bf421a1183101290e93d6c28bfe0bce4fbb6abd327b"} Jul 24 12:19:40.373267 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.373221 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jslk5" event={"ID":"cecbdba0-1239-44c4-ba86-cb670f0e7f6d","Type":"ContainerStarted","Data":"347e57c17e673e4ac5d5235a5e6017f2266c4290c81912f9695ae7e22d3c0598"} Jul 24 12:19:40.374602 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.374568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" event={"ID":"2647aee1-a03d-448d-9597-401ac14cf241","Type":"ContainerStarted","Data":"abe1c8f43087c1f8734a932ae1d3edbfbc4d457c202a596896d5fabd2b67b5a3"} Jul 24 12:19:40.605984 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.605951 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.606185 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.606001 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:40.606185 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.606097 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 12:19:40.606185 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.606098 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:40.606185 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.606162 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:41.606146151 +0000 UTC m=+37.891691814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:40.606185 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.606176 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:41.60617015 +0000 UTC m=+37.891715813 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : configmap "kube-rbac-proxy" not found Jul 24 12:19:40.807678 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.807639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:40.807847 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:40.807753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:40.807847 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.807795 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 12:19:40.807847 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.807816 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b56447786-85wbl: secret "image-registry-tls" not found Jul 24 12:19:40.807994 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.807868 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 12:19:40.807994 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.807871 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls podName:c58a1211-6ad4-4b3a-a037-de7bef79913b nodeName:}" failed. No retries permitted until 2026-07-24 12:19:42.807855184 +0000 UTC m=+39.093400851 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls") pod "image-registry-5b56447786-85wbl" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b") : secret "image-registry-tls" not found Jul 24 12:19:40.807994 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:40.807928 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls podName:7ac784d3-ac39-4697-8db8-90e5b096374f nodeName:}" failed. No retries permitted until 2026-07-24 12:19:42.807913033 +0000 UTC m=+39.093458696 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls") pod "dns-default-2g8z9" (UID: "7ac784d3-ac39-4697-8db8-90e5b096374f") : secret "dns-default-metrics-tls" not found Jul 24 12:19:41.381462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.381348 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="3bfefa9c23455e6c46f71bf421a1183101290e93d6c28bfe0bce4fbb6abd327b" exitCode=0 Jul 24 12:19:41.381462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.381407 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"3bfefa9c23455e6c46f71bf421a1183101290e93d6c28bfe0bce4fbb6abd327b"} Jul 24 12:19:41.616226 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.615889 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:41.616226 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.615950 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:41.616226 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:41.616115 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 12:19:41.616226 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:41.616174 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:43.616154604 +0000 UTC m=+39.901700269 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : configmap "kube-rbac-proxy" not found Jul 24 12:19:41.616655 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:41.616567 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:41.616655 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:41.616636 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:43.616619518 +0000 UTC m=+39.902165187 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:41.871509 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.871476 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-r8nwq"] Jul 24 12:19:41.873247 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.873217 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:41.875429 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.875396 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 24 12:19:41.875622 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.875601 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 24 12:19:41.875832 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.875811 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-jtgbv\"" Jul 24 12:19:41.875909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.875843 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 24 12:19:41.875972 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.875944 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 24 12:19:41.887278 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:41.886987 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-r8nwq"] Jul 24 12:19:42.019217 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.019155 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmmzl\" (UniqueName: \"kubernetes.io/projected/c53ef7d2-15e6-4d63-a02c-702050ad1572-kube-api-access-xmmzl\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.019388 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.019237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-cabundle\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.019445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.019374 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-key\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.069648 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.069616 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4blg2_598c1f25-29f8-4ce5-acf0-362bcc9c4854/dns-node-resolver/0.log" Jul 24 12:19:42.120484 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.120435 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmmzl\" (UniqueName: \"kubernetes.io/projected/c53ef7d2-15e6-4d63-a02c-702050ad1572-kube-api-access-xmmzl\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.120661 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.120490 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-cabundle\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.120661 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.120574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-key\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.121509 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.121477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-cabundle\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.125689 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.125663 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/c53ef7d2-15e6-4d63-a02c-702050ad1572-signing-key\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.131544 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.131517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmmzl\" (UniqueName: \"kubernetes.io/projected/c53ef7d2-15e6-4d63-a02c-702050ad1572-kube-api-access-xmmzl\") pod \"service-ca-658d7b7c78-r8nwq\" (UID: \"c53ef7d2-15e6-4d63-a02c-702050ad1572\") " pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.185461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.185369 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" Jul 24 12:19:42.385994 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.385956 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" event={"ID":"2647aee1-a03d-448d-9597-401ac14cf241","Type":"ContainerStarted","Data":"24e3f2da021e768256e9cc9e3c23f9c984bbc802e0c6dd83a7a3cc73bafdcec3"} Jul 24 12:19:42.386469 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.386026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" event={"ID":"2647aee1-a03d-448d-9597-401ac14cf241","Type":"ContainerStarted","Data":"daafb83cb4e845c742832186a06c79dcff5d9352d4f228de6a3f75c76d7b43fa"} Jul 24 12:19:42.389384 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.389354 2578 generic.go:356] "Generic (PLEG): container finished" podID="4cb9d1e1-a757-4039-850e-91715488a92d" containerID="6cb8510226d1a2465e3732a5be41523c0023d214b3ff1f6dfeaea6b89b41c92b" exitCode=0 Jul 24 12:19:42.389534 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.389397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerDied","Data":"6cb8510226d1a2465e3732a5be41523c0023d214b3ff1f6dfeaea6b89b41c92b"} Jul 24 12:19:42.402525 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.402455 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-6865c9967-j84gd" podStartSLOduration=2.204319763 podStartE2EDuration="3.402436233s" totalImagesPullingTime="1.19811647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:42.401790264 +0000 UTC m=+38.687335964" watchObservedRunningTime="2026-07-24 12:19:42.402436233 +0000 UTC m=+38.687981919" Jul 24 12:19:42.783523 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.783352 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-658d7b7c78-r8nwq"] Jul 24 12:19:42.787601 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:42.787576 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc53ef7d2_15e6_4d63_a02c_702050ad1572.slice/crio-16ceccdb46ba596873c7c1de27549d13c9f47e471f8de334a0f3fc5a45880bbc WatchSource:0}: Error finding container 16ceccdb46ba596873c7c1de27549d13c9f47e471f8de334a0f3fc5a45880bbc: Status 404 returned error can't find the container with id 16ceccdb46ba596873c7c1de27549d13c9f47e471f8de334a0f3fc5a45880bbc Jul 24 12:19:42.827077 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.827050 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:42.827248 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:42.827104 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:42.827248 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:42.827228 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 12:19:42.827248 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:42.827237 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 12:19:42.827248 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:42.827247 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b56447786-85wbl: secret "image-registry-tls" not found Jul 24 12:19:42.827391 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:42.827286 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls podName:7ac784d3-ac39-4697-8db8-90e5b096374f nodeName:}" failed. No retries permitted until 2026-07-24 12:19:46.827271795 +0000 UTC m=+43.112817459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls") pod "dns-default-2g8z9" (UID: "7ac784d3-ac39-4697-8db8-90e5b096374f") : secret "dns-default-metrics-tls" not found Jul 24 12:19:42.827391 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:42.827299 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls podName:c58a1211-6ad4-4b3a-a037-de7bef79913b nodeName:}" failed. No retries permitted until 2026-07-24 12:19:46.827293415 +0000 UTC m=+43.112839078 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls") pod "image-registry-5b56447786-85wbl" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b") : secret "image-registry-tls" not found Jul 24 12:19:43.068664 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.068640 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkz6p_e1086883-e355-4c97-9849-9efa8b112d13/node-ca/0.log" Jul 24 12:19:43.392546 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.392509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" event={"ID":"c53ef7d2-15e6-4d63-a02c-702050ad1572","Type":"ContainerStarted","Data":"16ceccdb46ba596873c7c1de27549d13c9f47e471f8de334a0f3fc5a45880bbc"} Jul 24 12:19:43.395622 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.395598 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" event={"ID":"4cb9d1e1-a757-4039-850e-91715488a92d","Type":"ContainerStarted","Data":"8fbefe78667c4483e9da1d555911525226a2652c7ff3ea8b48299adf4a59a94d"} Jul 24 12:19:43.396833 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.396808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-jslk5" event={"ID":"cecbdba0-1239-44c4-ba86-cb670f0e7f6d","Type":"ContainerStarted","Data":"47d6c585dbc010d1abff4900b9d363ad4bdeaca68fb06f442720dd37f33448cd"} Jul 24 12:19:43.421936 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.421889 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-wf5j6" podStartSLOduration=8.654448111 podStartE2EDuration="39.421873804s" totalImagesPullingTime="30.767425693s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:43.42129264 +0000 UTC m=+39.706838345" watchObservedRunningTime="2026-07-24 12:19:43.421873804 +0000 UTC m=+39.707419487" Jul 24 12:19:43.438418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.438368 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-jslk5" podStartSLOduration=1.736387637 podStartE2EDuration="4.43835265s" totalImagesPullingTime="2.701965013s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:43.437375498 +0000 UTC m=+39.722921182" watchObservedRunningTime="2026-07-24 12:19:43.43835265 +0000 UTC m=+39.723898318" Jul 24 12:19:43.634229 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.634175 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:43.634418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.634246 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:43.634418 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:43.634346 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:43.634418 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:43.634355 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 12:19:43.634622 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:43.634421 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:47.634401671 +0000 UTC m=+43.919947335 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:43.634622 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:43.634442 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:47.634431254 +0000 UTC m=+43.919976918 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : configmap "kube-rbac-proxy" not found Jul 24 12:19:43.868545 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:43.868497 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jslk5_cecbdba0-1239-44c4-ba86-cb670f0e7f6d/serve-healthcheck-canary/0.log" Jul 24 12:19:44.268693 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:44.268598 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-j84gd_2647aee1-a03d-448d-9597-401ac14cf241/migrator/0.log" Jul 24 12:19:44.468550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:44.468507 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-j84gd_2647aee1-a03d-448d-9597-401ac14cf241/graceful-termination/0.log" Jul 24 12:19:45.403302 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:45.403270 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" event={"ID":"c53ef7d2-15e6-4d63-a02c-702050ad1572","Type":"ContainerStarted","Data":"19a637029ab18cf3da6a99dceceebaa1dbb5a9ad0bb33bb2942c9b971b0b4c7f"} Jul 24 12:19:45.417304 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:45.417259 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-658d7b7c78-r8nwq" podStartSLOduration=1.9981897320000002 podStartE2EDuration="4.417241706s" totalImagesPullingTime="2.419051974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:45.416816989 +0000 UTC m=+41.702362695" watchObservedRunningTime="2026-07-24 12:19:45.417241706 +0000 UTC m=+41.702787371" Jul 24 12:19:46.859663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:46.859622 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:46.860107 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:46.859719 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:46.860107 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:46.859909 2578 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 24 12:19:46.860107 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:46.859925 2578 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5b56447786-85wbl: secret "image-registry-tls" not found Jul 24 12:19:46.860107 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:46.859983 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls podName:c58a1211-6ad4-4b3a-a037-de7bef79913b nodeName:}" failed. No retries permitted until 2026-07-24 12:19:54.859964821 +0000 UTC m=+51.145510497 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls") pod "image-registry-5b56447786-85wbl" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b") : secret "image-registry-tls" not found Jul 24 12:19:46.860428 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:46.860409 2578 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 24 12:19:46.860487 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:46.860475 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls podName:7ac784d3-ac39-4697-8db8-90e5b096374f nodeName:}" failed. No retries permitted until 2026-07-24 12:19:54.860458777 +0000 UTC m=+51.146004443 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls") pod "dns-default-2g8z9" (UID: "7ac784d3-ac39-4697-8db8-90e5b096374f") : secret "dns-default-metrics-tls" not found Jul 24 12:19:47.668725 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:47.668687 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:47.668925 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:47.668736 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:47.668925 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:47.668863 2578 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 24 12:19:47.668925 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:47.668863 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:47.669083 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:47.668930 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:55.668911783 +0000 UTC m=+51.954457460 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : configmap "kube-rbac-proxy" not found Jul 24 12:19:47.669083 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:47.668945 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:19:55.668939046 +0000 UTC m=+51.954484709 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:54.928235 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.928181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:54.928675 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.928261 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:54.931477 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.931443 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/7ac784d3-ac39-4697-8db8-90e5b096374f-metrics-tls\") pod \"dns-default-2g8z9\" (UID: \"7ac784d3-ac39-4697-8db8-90e5b096374f\") " pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:54.931658 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.931636 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"image-registry-5b56447786-85wbl\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:54.969643 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.969614 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:54.985777 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:54.985752 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:55.130610 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.130574 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:19:55.134243 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:55.134218 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58a1211_6ad4_4b3a_a037_de7bef79913b.slice/crio-bea9722ea174734ee11b6ae96047fe45ea7a738150507d1da7d0c033b19f1e65 WatchSource:0}: Error finding container bea9722ea174734ee11b6ae96047fe45ea7a738150507d1da7d0c033b19f1e65: Status 404 returned error can't find the container with id bea9722ea174734ee11b6ae96047fe45ea7a738150507d1da7d0c033b19f1e65 Jul 24 12:19:55.150677 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.149277 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-2g8z9"] Jul 24 12:19:55.154924 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:19:55.154895 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7ac784d3_ac39_4697_8db8_90e5b096374f.slice/crio-2f249a90554552ee57a53f7de6f62f24a6c9556fc8144d90cce577fdd43c467e WatchSource:0}: Error finding container 2f249a90554552ee57a53f7de6f62f24a6c9556fc8144d90cce577fdd43c467e: Status 404 returned error can't find the container with id 2f249a90554552ee57a53f7de6f62f24a6c9556fc8144d90cce577fdd43c467e Jul 24 12:19:55.429474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.429379 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2g8z9" event={"ID":"7ac784d3-ac39-4697-8db8-90e5b096374f","Type":"ContainerStarted","Data":"2f249a90554552ee57a53f7de6f62f24a6c9556fc8144d90cce577fdd43c467e"} Jul 24 12:19:55.430906 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.430872 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b56447786-85wbl" event={"ID":"c58a1211-6ad4-4b3a-a037-de7bef79913b","Type":"ContainerStarted","Data":"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a"} Jul 24 12:19:55.431040 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.430925 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b56447786-85wbl" event={"ID":"c58a1211-6ad4-4b3a-a037-de7bef79913b","Type":"ContainerStarted","Data":"bea9722ea174734ee11b6ae96047fe45ea7a738150507d1da7d0c033b19f1e65"} Jul 24 12:19:55.431040 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.431011 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:19:55.450380 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.450324 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b56447786-85wbl" podStartSLOduration=25.45030704 podStartE2EDuration="25.45030704s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:55.449098589 +0000 UTC m=+51.734644272" watchObservedRunningTime="2026-07-24 12:19:55.45030704 +0000 UTC m=+51.735852724" Jul 24 12:19:55.735484 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.735447 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:55.735645 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.735495 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:55.735645 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:55.735602 2578 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 24 12:19:55.735713 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:19:55.735677 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls podName:11bf7252-0597-488c-a6f0-4711e2e20ae0 nodeName:}" failed. No retries permitted until 2026-07-24 12:20:11.735659102 +0000 UTC m=+68.021204766 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls") pod "insights-runtime-extractor-zvrkq" (UID: "11bf7252-0597-488c-a6f0-4711e2e20ae0") : secret "insights-runtime-extractor-tls" not found Jul 24 12:19:55.736107 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:55.736088 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/11bf7252-0597-488c-a6f0-4711e2e20ae0-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:19:57.439297 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:57.439262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2g8z9" event={"ID":"7ac784d3-ac39-4697-8db8-90e5b096374f","Type":"ContainerStarted","Data":"588d2d4364b303534ce32ca85328fa819a3acd092ce950633220e4fb96500603"} Jul 24 12:19:58.444793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:58.444757 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-2g8z9" event={"ID":"7ac784d3-ac39-4697-8db8-90e5b096374f","Type":"ContainerStarted","Data":"9ee8fbebd309517a42754efd958bbd59325a3ec57995d84dc2c2a8ecc7263043"} Jul 24 12:19:58.445273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:58.444880 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-2g8z9" Jul 24 12:19:58.468212 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:19:58.468140 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-2g8z9" podStartSLOduration=17.390775286 podStartE2EDuration="19.468125973s" totalImagesPullingTime="2.077350687s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:19:58.467323409 +0000 UTC m=+54.752869133" watchObservedRunningTime="2026-07-24 12:19:58.468125973 +0000 UTC m=+54.753671659" Jul 24 12:20:03.707132 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.707098 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-7c97dc4555-t5csq"] Jul 24 12:20:03.741163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.741128 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:20:03.741163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.741167 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-t5csq"] Jul 24 12:20:03.741401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.741294 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:03.743235 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.743207 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 24 12:20:03.743529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.743509 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qb55h\"" Jul 24 12:20:03.743609 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.743532 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 24 12:20:03.794415 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.794383 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4ldk9\" (UniqueName: \"kubernetes.io/projected/f013bb68-2da6-44c1-a26c-c8f097390f8f-kube-api-access-4ldk9\") pod \"downloads-7c97dc4555-t5csq\" (UID: \"f013bb68-2da6-44c1-a26c-c8f097390f8f\") " pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:03.816923 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.816891 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7c9d66689c-9wthm"] Jul 24 12:20:03.841926 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.841901 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9d66689c-9wthm"] Jul 24 12:20:03.842073 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.841990 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895055 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895019 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z445k\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-kube-api-access-z445k\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895249 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895074 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-image-registry-private-configuration\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895249 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895093 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de704828-fa9e-41c7-a646-cc1d0db04e31-ca-trust-extracted\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895249 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895141 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-bound-sa-token\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895249 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-installation-pull-secrets\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895249 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4ldk9\" (UniqueName: \"kubernetes.io/projected/f013bb68-2da6-44c1-a26c-c8f097390f8f-kube-api-access-4ldk9\") pod \"downloads-7c97dc4555-t5csq\" (UID: \"f013bb68-2da6-44c1-a26c-c8f097390f8f\") " pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:03.895511 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895309 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-trusted-ca\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895511 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895357 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-tls\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.895511 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.895423 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-certificates\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.908923 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.908890 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4ldk9\" (UniqueName: \"kubernetes.io/projected/f013bb68-2da6-44c1-a26c-c8f097390f8f-kube-api-access-4ldk9\") pod \"downloads-7c97dc4555-t5csq\" (UID: \"f013bb68-2da6-44c1-a26c-c8f097390f8f\") " pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:03.996134 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-image-registry-private-configuration\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996134 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996089 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de704828-fa9e-41c7-a646-cc1d0db04e31-ca-trust-extracted\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996134 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996107 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-bound-sa-token\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996239 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-installation-pull-secrets\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996301 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-trusted-ca\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996337 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-tls\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996418 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-certificates\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996450 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z445k\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-kube-api-access-z445k\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.996630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.996517 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/de704828-fa9e-41c7-a646-cc1d0db04e31-ca-trust-extracted\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.997108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.997085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-certificates\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.997309 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.997292 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/de704828-fa9e-41c7-a646-cc1d0db04e31-trusted-ca\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.999016 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.998987 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-registry-tls\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.999134 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.999092 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-image-registry-private-configuration\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:03.999246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:03.999224 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/de704828-fa9e-41c7-a646-cc1d0db04e31-installation-pull-secrets\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:04.004585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.004560 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-bound-sa-token\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:04.004716 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.004700 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z445k\" (UniqueName: \"kubernetes.io/projected/de704828-fa9e-41c7-a646-cc1d0db04e31-kube-api-access-z445k\") pod \"image-registry-7c9d66689c-9wthm\" (UID: \"de704828-fa9e-41c7-a646-cc1d0db04e31\") " pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:04.053105 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.053081 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-qb55h\"" Jul 24 12:20:04.062519 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.062489 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:04.150997 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.150946 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:04.222220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.222116 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-7c97dc4555-t5csq"] Jul 24 12:20:04.227114 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:04.226907 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf013bb68_2da6_44c1_a26c_c8f097390f8f.slice/crio-dd9cc63bfec1485a632b6da06121c63c644bc66912047c06e0a0a386f3a24e03 WatchSource:0}: Error finding container dd9cc63bfec1485a632b6da06121c63c644bc66912047c06e0a0a386f3a24e03: Status 404 returned error can't find the container with id dd9cc63bfec1485a632b6da06121c63c644bc66912047c06e0a0a386f3a24e03 Jul 24 12:20:04.303067 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.303034 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7c9d66689c-9wthm"] Jul 24 12:20:04.306047 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:04.306022 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde704828_fa9e_41c7_a646_cc1d0db04e31.slice/crio-b809a34d5bda5d71c2ce1071cf3b74ca963e5c58a68619f1df1d1ed948938534 WatchSource:0}: Error finding container b809a34d5bda5d71c2ce1071cf3b74ca963e5c58a68619f1df1d1ed948938534: Status 404 returned error can't find the container with id b809a34d5bda5d71c2ce1071cf3b74ca963e5c58a68619f1df1d1ed948938534 Jul 24 12:20:04.468694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.468647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" event={"ID":"de704828-fa9e-41c7-a646-cc1d0db04e31","Type":"ContainerStarted","Data":"030fe5ad871ee47773e9de1b556753a7840c5bd8745c53a34d22796144258615"} Jul 24 12:20:04.468887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.468740 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" event={"ID":"de704828-fa9e-41c7-a646-cc1d0db04e31","Type":"ContainerStarted","Data":"b809a34d5bda5d71c2ce1071cf3b74ca963e5c58a68619f1df1d1ed948938534"} Jul 24 12:20:04.468887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.468766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:04.469873 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.469850 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-t5csq" event={"ID":"f013bb68-2da6-44c1-a26c-c8f097390f8f","Type":"ContainerStarted","Data":"dd9cc63bfec1485a632b6da06121c63c644bc66912047c06e0a0a386f3a24e03"} Jul 24 12:20:04.484678 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:04.484633 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" podStartSLOduration=1.48461843 podStartE2EDuration="1.48461843s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:20:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:04.484350928 +0000 UTC m=+60.769896612" watchObservedRunningTime="2026-07-24 12:20:04.48461843 +0000 UTC m=+60.770164114" Jul 24 12:20:05.365563 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.365498 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-mg9vw" Jul 24 12:20:05.899430 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.899395 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:20:05.902414 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.902386 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:05.904578 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.904550 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 12:20:05.904578 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.904573 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 12:20:05.904765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.904626 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 12:20:05.905087 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.905062 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l6rpx\"" Jul 24 12:20:05.905439 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.905417 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 12:20:05.905693 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.905677 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 12:20:05.915480 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:05.915458 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:20:06.013556 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013519 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.013734 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.013734 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013636 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.013734 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.013897 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srlsb\" (UniqueName: \"kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.013897 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.013830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115090 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115051 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115327 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115327 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115166 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115327 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115215 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115327 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115251 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srlsb\" (UniqueName: \"kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.115327 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.115309 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.116064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.116042 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.116064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.116054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.116211 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.116054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.118005 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.117981 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.118486 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.118469 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.122692 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.122666 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srlsb\" (UniqueName: \"kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb\") pod \"console-d84d8bc9-hxlmh\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.200877 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.200832 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g"] Jul 24 12:20:06.205111 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.205085 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:06.206894 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.206874 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 24 12:20:06.207174 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.207158 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-7hb67\"" Jul 24 12:20:06.210657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.210637 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g"] Jul 24 12:20:06.212539 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.212517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:06.316965 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.316928 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b5c6dd55-ca59-443d-9467-d771aecc7c5d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-z2z9g\" (UID: \"b5c6dd55-ca59-443d-9467-d771aecc7c5d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:06.354836 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.354803 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:20:06.358622 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:06.358593 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e65d668_afad_42b8_b767_1ee7baa2d269.slice/crio-60f35c4f002cb3879be8d2a88296132be2435eb2e2b6decceef06de7eebc8a0f WatchSource:0}: Error finding container 60f35c4f002cb3879be8d2a88296132be2435eb2e2b6decceef06de7eebc8a0f: Status 404 returned error can't find the container with id 60f35c4f002cb3879be8d2a88296132be2435eb2e2b6decceef06de7eebc8a0f Jul 24 12:20:06.418348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.418316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b5c6dd55-ca59-443d-9467-d771aecc7c5d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-z2z9g\" (UID: \"b5c6dd55-ca59-443d-9467-d771aecc7c5d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:06.421301 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.421266 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/b5c6dd55-ca59-443d-9467-d771aecc7c5d-tls-certificates\") pod \"prometheus-operator-admission-webhook-7f8d78f54b-z2z9g\" (UID: \"b5c6dd55-ca59-443d-9467-d771aecc7c5d\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:06.478465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.478383 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d84d8bc9-hxlmh" event={"ID":"4e65d668-afad-42b8-b767-1ee7baa2d269","Type":"ContainerStarted","Data":"60f35c4f002cb3879be8d2a88296132be2435eb2e2b6decceef06de7eebc8a0f"} Jul 24 12:20:06.516835 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.516790 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:06.665892 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:06.665858 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g"] Jul 24 12:20:06.669387 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:06.669344 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5c6dd55_ca59_443d_9467_d771aecc7c5d.slice/crio-447cf77e85eb035aea991b90feab697572739153ef035748d8310ceef719ae84 WatchSource:0}: Error finding container 447cf77e85eb035aea991b90feab697572739153ef035748d8310ceef719ae84: Status 404 returned error can't find the container with id 447cf77e85eb035aea991b90feab697572739153ef035748d8310ceef719ae84 Jul 24 12:20:07.484169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:07.484122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" event={"ID":"b5c6dd55-ca59-443d-9467-d771aecc7c5d","Type":"ContainerStarted","Data":"447cf77e85eb035aea991b90feab697572739153ef035748d8310ceef719ae84"} Jul 24 12:20:08.451839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.451754 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-2g8z9" Jul 24 12:20:08.497503 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.497463 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" event={"ID":"b5c6dd55-ca59-443d-9467-d771aecc7c5d","Type":"ContainerStarted","Data":"6c05e151013c5270337cc1ec97068ffa8264b7b99b4a4280b706167830407e8a"} Jul 24 12:20:08.497936 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.497860 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:08.503626 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.503600 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" Jul 24 12:20:08.512210 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.512140 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-7f8d78f54b-z2z9g" podStartSLOduration=0.982771385 podStartE2EDuration="2.51212541s" totalImagesPullingTime="1.529354025s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:20:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:08.511839256 +0000 UTC m=+64.797384963" watchObservedRunningTime="2026-07-24 12:20:08.51212541 +0000 UTC m=+64.797671143" Jul 24 12:20:08.949027 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.948988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:20:08.949252 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.949075 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:20:08.949252 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.949108 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:20:08.951168 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.951139 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 24 12:20:08.951305 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.951140 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 24 12:20:08.951714 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.951547 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 24 12:20:08.961420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.961399 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 24 12:20:08.962656 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.962627 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7dc1ac0f-2573-444a-ad36-c0d4be8b025a-metrics-certs\") pod \"network-metrics-daemon-5ss9t\" (UID: \"7dc1ac0f-2573-444a-ad36-c0d4be8b025a\") " pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:20:08.963346 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.963322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/bf78f542-b932-49f4-9476-ed5cccaaaccd-original-pull-secret\") pod \"global-pull-secret-syncer-w7wlm\" (UID: \"bf78f542-b932-49f4-9476-ed5cccaaaccd\") " pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:20:08.973162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:08.973135 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wq2p5\" (UniqueName: \"kubernetes.io/projected/8ec8dd0a-5096-44cc-84d1-b126e867dae2-kube-api-access-wq2p5\") pod \"network-check-target-ln2m5\" (UID: \"8ec8dd0a-5096-44cc-84d1-b126e867dae2\") " pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:20:09.157040 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.157003 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-k4xp5\"" Jul 24 12:20:09.162833 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.162800 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w7wlm" Jul 24 12:20:09.165943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.165910 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:20:09.169850 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.169829 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zhxrp\"" Jul 24 12:20:09.178272 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.178247 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-5ss9t" Jul 24 12:20:09.803882 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.803850 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w7wlm"] Jul 24 12:20:09.808259 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:09.808216 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf78f542_b932_49f4_9476_ed5cccaaaccd.slice/crio-fe19b0c3a7c9721372b74b3262c191efbb175634b815a09a798412c9a610757d WatchSource:0}: Error finding container fe19b0c3a7c9721372b74b3262c191efbb175634b815a09a798412c9a610757d: Status 404 returned error can't find the container with id fe19b0c3a7c9721372b74b3262c191efbb175634b815a09a798412c9a610757d Jul 24 12:20:09.830435 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.830356 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ln2m5"] Jul 24 12:20:09.834141 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:09.834110 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ec8dd0a_5096_44cc_84d1_b126e867dae2.slice/crio-a1f6fa94d0d0cd86d4d0172e832ad827d784c1d3575a2a005acbe5968d39d63b WatchSource:0}: Error finding container a1f6fa94d0d0cd86d4d0172e832ad827d784c1d3575a2a005acbe5968d39d63b: Status 404 returned error can't find the container with id a1f6fa94d0d0cd86d4d0172e832ad827d784c1d3575a2a005acbe5968d39d63b Jul 24 12:20:09.851437 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:09.851307 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-5ss9t"] Jul 24 12:20:09.855290 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:09.855262 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc1ac0f_2573_444a_ad36_c0d4be8b025a.slice/crio-6a0aee6100b5e530d3d2a06dab82bb7a8f185e55fd188223ad7ac7a4574efb04 WatchSource:0}: Error finding container 6a0aee6100b5e530d3d2a06dab82bb7a8f185e55fd188223ad7ac7a4574efb04: Status 404 returned error can't find the container with id 6a0aee6100b5e530d3d2a06dab82bb7a8f185e55fd188223ad7ac7a4574efb04 Jul 24 12:20:10.509087 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:10.508650 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d84d8bc9-hxlmh" event={"ID":"4e65d668-afad-42b8-b767-1ee7baa2d269","Type":"ContainerStarted","Data":"ff34ddad7e507d268d71c33c016a2ecafd4f95272b1451e74ff5e98815464c80"} Jul 24 12:20:10.510959 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:10.510929 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ln2m5" event={"ID":"8ec8dd0a-5096-44cc-84d1-b126e867dae2","Type":"ContainerStarted","Data":"a1f6fa94d0d0cd86d4d0172e832ad827d784c1d3575a2a005acbe5968d39d63b"} Jul 24 12:20:10.513457 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:10.513429 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ss9t" event={"ID":"7dc1ac0f-2573-444a-ad36-c0d4be8b025a","Type":"ContainerStarted","Data":"6a0aee6100b5e530d3d2a06dab82bb7a8f185e55fd188223ad7ac7a4574efb04"} Jul 24 12:20:10.515147 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:10.515117 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w7wlm" event={"ID":"bf78f542-b932-49f4-9476-ed5cccaaaccd","Type":"ContainerStarted","Data":"fe19b0c3a7c9721372b74b3262c191efbb175634b815a09a798412c9a610757d"} Jul 24 12:20:10.525966 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:10.525907 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-d84d8bc9-hxlmh" podStartSLOduration=2.200417752 podStartE2EDuration="5.525877859s" totalImagesPullingTime="3.325460107s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:20:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:10.525541342 +0000 UTC m=+66.811087028" watchObservedRunningTime="2026-07-24 12:20:10.525877859 +0000 UTC m=+66.811423542" Jul 24 12:20:11.524874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:11.524764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ss9t" event={"ID":"7dc1ac0f-2573-444a-ad36-c0d4be8b025a","Type":"ContainerStarted","Data":"3c3b35ca26e6aa01b76c6482f4eab54fa4110d9e6238e609c0ac1a69c54b890c"} Jul 24 12:20:11.784011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:11.783459 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:20:11.787775 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:11.787726 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/11bf7252-0597-488c-a6f0-4711e2e20ae0-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-zvrkq\" (UID: \"11bf7252-0597-488c-a6f0-4711e2e20ae0\") " pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:20:12.003276 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.003157 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-sd4k9\"" Jul 24 12:20:12.011169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.011134 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-zvrkq" Jul 24 12:20:12.196657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.196615 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-zvrkq"] Jul 24 12:20:12.200800 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:12.200766 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11bf7252_0597_488c_a6f0_4711e2e20ae0.slice/crio-865450d73150a755a22d6b9a6e76f507d8461e5b868356d085e2dd550a2edabe WatchSource:0}: Error finding container 865450d73150a755a22d6b9a6e76f507d8461e5b868356d085e2dd550a2edabe: Status 404 returned error can't find the container with id 865450d73150a755a22d6b9a6e76f507d8461e5b868356d085e2dd550a2edabe Jul 24 12:20:12.532038 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.531934 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-5ss9t" event={"ID":"7dc1ac0f-2573-444a-ad36-c0d4be8b025a","Type":"ContainerStarted","Data":"1a7685dbc6a80d19b1a98cef7d57e25763914441f59fa305c36691700f5bc6d1"} Jul 24 12:20:12.534549 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.534495 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvrkq" event={"ID":"11bf7252-0597-488c-a6f0-4711e2e20ae0","Type":"ContainerStarted","Data":"865450d73150a755a22d6b9a6e76f507d8461e5b868356d085e2dd550a2edabe"} Jul 24 12:20:12.548936 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:12.548869 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-5ss9t" podStartSLOduration=67.092080959 podStartE2EDuration="1m8.548848169s" totalImagesPullingTime="1.45676721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:12.546077705 +0000 UTC m=+68.831623392" watchObservedRunningTime="2026-07-24 12:20:12.548848169 +0000 UTC m=+68.834393856" Jul 24 12:20:13.747639 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:13.747588 2578 patch_prober.go:28] interesting pod/image-registry-5b56447786-85wbl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 24 12:20:13.748054 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:13.747659 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-5b56447786-85wbl" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 12:20:16.213568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:16.213398 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:16.213568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:16.213478 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:16.219894 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:16.219865 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:16.556487 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:16.556453 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:20:18.641907 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.640601 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lxbzv"] Jul 24 12:20:18.644395 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.644365 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.646697 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.646638 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-wc5gs\"" Jul 24 12:20:18.646697 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.646688 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 24 12:20:18.647028 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.647012 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 24 12:20:18.647227 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.647213 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 24 12:20:18.647352 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.647335 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 24 12:20:18.647401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.647380 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 24 12:20:18.647451 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.647400 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 24 12:20:18.744660 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744593 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-textfile\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744660 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744700 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-wtmp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744726 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-sys\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744817 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-tls\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.744909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pnpp\" (UniqueName: \"kubernetes.io/projected/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-kube-api-access-5pnpp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.745326 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744920 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-metrics-client-ca\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.745326 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.744958 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-root\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849463 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-tls\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849506 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pnpp\" (UniqueName: \"kubernetes.io/projected/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-kube-api-access-5pnpp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-metrics-client-ca\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849571 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-root\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849605 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-textfile\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849639 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849670 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-wtmp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849697 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-sys\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.849722 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.850317 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-accelerators-collector-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.850876 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-textfile\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.851625 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-metrics-client-ca\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.851689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-root\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.851803 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-wtmp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.852908 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.852616 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-sys\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.857879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.857496 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.857879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.857834 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-node-exporter-tls\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.866703 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.866657 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pnpp\" (UniqueName: \"kubernetes.io/projected/00a7b9f2-bef2-4620-b1e9-e6baf4caea22-kube-api-access-5pnpp\") pod \"node-exporter-lxbzv\" (UID: \"00a7b9f2-bef2-4620-b1e9-e6baf4caea22\") " pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:18.957401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:18.957319 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lxbzv" Jul 24 12:20:23.006705 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:20:23.006664 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00a7b9f2_bef2_4620_b1e9_e6baf4caea22.slice/crio-f8ebe35e51a27000b127593ee5e7c7660cf0d39cff22f68aaee33a78ae3a23f4 WatchSource:0}: Error finding container f8ebe35e51a27000b127593ee5e7c7660cf0d39cff22f68aaee33a78ae3a23f4: Status 404 returned error can't find the container with id f8ebe35e51a27000b127593ee5e7c7660cf0d39cff22f68aaee33a78ae3a23f4 Jul 24 12:20:23.589220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.586275 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-7c97dc4555-t5csq" event={"ID":"f013bb68-2da6-44c1-a26c-c8f097390f8f","Type":"ContainerStarted","Data":"806fa829c97bd2ac7d450a8a981faf6002a724bea2d70ada6ac1b432bf6bce75"} Jul 24 12:20:23.589642 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.589612 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:23.592114 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.592062 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ln2m5" event={"ID":"8ec8dd0a-5096-44cc-84d1-b126e867dae2","Type":"ContainerStarted","Data":"1a2e42ad7ae53c679b4cef4ad02c8dfb1f49d95392a0c5e653ffdbc403e98a5b"} Jul 24 12:20:23.592347 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.592331 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:20:23.594279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.593306 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-7c97dc4555-t5csq" Jul 24 12:20:23.594279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.594167 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w7wlm" event={"ID":"bf78f542-b932-49f4-9476-ed5cccaaaccd","Type":"ContainerStarted","Data":"89d006f59b2286bdf6e8cd41d789dc581eae2fa808e07bf34017b94edb097e22"} Jul 24 12:20:23.596358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.596329 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxbzv" event={"ID":"00a7b9f2-bef2-4620-b1e9-e6baf4caea22","Type":"ContainerStarted","Data":"f8ebe35e51a27000b127593ee5e7c7660cf0d39cff22f68aaee33a78ae3a23f4"} Jul 24 12:20:23.618134 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.618056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-7c97dc4555-t5csq" podStartSLOduration=1.784565234 podStartE2EDuration="20.618035183s" totalImagesPullingTime="18.833469949s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:23.604576693 +0000 UTC m=+79.890122380" watchObservedRunningTime="2026-07-24 12:20:23.618035183 +0000 UTC m=+79.903580870" Jul 24 12:20:23.635993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.634564 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ln2m5" podStartSLOduration=66.46809321 podStartE2EDuration="1m19.634543065s" totalImagesPullingTime="13.166449855s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:23.618948802 +0000 UTC m=+79.904494487" watchObservedRunningTime="2026-07-24 12:20:23.634543065 +0000 UTC m=+79.920088754" Jul 24 12:20:23.650098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.649719 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-w7wlm" podStartSLOduration=66.457619342 podStartE2EDuration="1m19.64969776s" totalImagesPullingTime="13.192078418s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:23.649645873 +0000 UTC m=+79.935191560" watchObservedRunningTime="2026-07-24 12:20:23.64969776 +0000 UTC m=+79.935243469" Jul 24 12:20:23.747443 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:23.747230 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:20:25.479994 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:25.479818 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-7c9d66689c-9wthm" Jul 24 12:20:28.630447 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:28.630347 2578 generic.go:356] "Generic (PLEG): container finished" podID="00a7b9f2-bef2-4620-b1e9-e6baf4caea22" containerID="ad5e97de043403038504e2b0ddb5bf24ec75513cf81c1895a736c2945b40e4cc" exitCode=0 Jul 24 12:20:28.630447 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:28.630396 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxbzv" event={"ID":"00a7b9f2-bef2-4620-b1e9-e6baf4caea22","Type":"ContainerDied","Data":"ad5e97de043403038504e2b0ddb5bf24ec75513cf81c1895a736c2945b40e4cc"} Jul 24 12:20:28.632595 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:28.632526 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvrkq" event={"ID":"11bf7252-0597-488c-a6f0-4711e2e20ae0","Type":"ContainerStarted","Data":"9fb807027d0d76ebaed46d8e368f2701be55cab9da3c6a6d1389e789b3118e26"} Jul 24 12:20:28.766465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:28.766175 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5b56447786-85wbl" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerName="registry" containerID="cri-o://25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a" gracePeriod=30 Jul 24 12:20:29.061812 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.061775 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:20:29.245552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245470 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245520 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245565 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-htdph\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245600 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245629 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245690 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.245780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.245722 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") pod \"c58a1211-6ad4-4b3a-a037-de7bef79913b\" (UID: \"c58a1211-6ad4-4b3a-a037-de7bef79913b\") " Jul 24 12:20:29.246364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.246325 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:20:29.246865 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.246827 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:20:29.248879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.248826 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:20:29.248979 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.248921 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph" (OuterVolumeSpecName: "kube-api-access-htdph") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "kube-api-access-htdph". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:20:29.249175 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.249100 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:20:29.249346 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.249316 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:20:29.249558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.249529 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:20:29.256633 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.256603 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "c58a1211-6ad4-4b3a-a037-de7bef79913b" (UID: "c58a1211-6ad4-4b3a-a037-de7bef79913b"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:20:29.347032 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.346992 2578 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/c58a1211-6ad4-4b3a-a037-de7bef79913b-ca-trust-extracted\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347032 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347031 2578 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-bound-sa-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347042 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-htdph\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-kube-api-access-htdph\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347053 2578 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-image-registry-private-configuration\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347063 2578 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-certificates\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347071 2578 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/c58a1211-6ad4-4b3a-a037-de7bef79913b-installation-pull-secrets\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347079 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c58a1211-6ad4-4b3a-a037-de7bef79913b-trusted-ca\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.347288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.347087 2578 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/c58a1211-6ad4-4b3a-a037-de7bef79913b-registry-tls\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:20:29.639656 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.639580 2578 generic.go:356] "Generic (PLEG): container finished" podID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerID="25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a" exitCode=0 Jul 24 12:20:29.640351 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.639658 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b56447786-85wbl" Jul 24 12:20:29.640351 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.639910 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b56447786-85wbl" event={"ID":"c58a1211-6ad4-4b3a-a037-de7bef79913b","Type":"ContainerDied","Data":"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a"} Jul 24 12:20:29.640351 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.639960 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b56447786-85wbl" event={"ID":"c58a1211-6ad4-4b3a-a037-de7bef79913b","Type":"ContainerDied","Data":"bea9722ea174734ee11b6ae96047fe45ea7a738150507d1da7d0c033b19f1e65"} Jul 24 12:20:29.640351 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.639990 2578 scope.go:117] "RemoveContainer" containerID="25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a" Jul 24 12:20:29.650776 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.649888 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxbzv" event={"ID":"00a7b9f2-bef2-4620-b1e9-e6baf4caea22","Type":"ContainerStarted","Data":"90d5fa7d8709e718a9ba9ff7a3b8296285a1dc5d14c018c31339be7ac23e5066"} Jul 24 12:20:29.650776 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.649945 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lxbzv" event={"ID":"00a7b9f2-bef2-4620-b1e9-e6baf4caea22","Type":"ContainerStarted","Data":"d7cb31f754c2b3a3fdbac23f3b70979cf15172d676bf504f177a541bc4fc5ddd"} Jul 24 12:20:29.658215 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.658163 2578 scope.go:117] "RemoveContainer" containerID="25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a" Jul 24 12:20:29.658689 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:20:29.658666 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a\": container with ID starting with 25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a not found: ID does not exist" containerID="25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a" Jul 24 12:20:29.658773 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.658699 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a"} err="failed to get container status \"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a\": rpc error: code = NotFound desc = could not find container \"25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a\": container with ID starting with 25db1abea604d3250da89e24c48b069f0f58aaab8f5581e73dc57a267720f68a not found: ID does not exist" Jul 24 12:20:29.669494 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.669449 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lxbzv" podStartSLOduration=6.238955148 podStartE2EDuration="11.669434573s" totalImagesPullingTime="5.324105999s" totalInitContainerRuntime="106.373426ms" isStatefulPod=true podCreationTimestamp="2026-07-24 12:20:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:29.667919264 +0000 UTC m=+85.953464968" watchObservedRunningTime="2026-07-24 12:20:29.669434573 +0000 UTC m=+85.954980257" Jul 24 12:20:29.681998 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.681967 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:20:29.686815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:29.686785 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5b56447786-85wbl"] Jul 24 12:20:30.150517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:30.150110 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" path="/var/lib/kubelet/pods/c58a1211-6ad4-4b3a-a037-de7bef79913b/volumes" Jul 24 12:20:34.673824 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:34.673783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvrkq" event={"ID":"11bf7252-0597-488c-a6f0-4711e2e20ae0","Type":"ContainerStarted","Data":"5e2f11c7c1cdd562cb261798cc1861b3e3fb269a23df03aac62e74ef9c70d6fa"} Jul 24 12:20:38.689943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:38.689903 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-zvrkq" event={"ID":"11bf7252-0597-488c-a6f0-4711e2e20ae0","Type":"ContainerStarted","Data":"6d0813fdabbc021b0303e2aecd42888a56c076e95d30a853d764f0bc7c55d8b9"} Jul 24 12:20:38.711730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:38.711679 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-zvrkq" podStartSLOduration=34.152819422 podStartE2EDuration="59.711662623s" totalImagesPullingTime="25.558843201s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 12:19:39 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:20:38.710519357 +0000 UTC m=+94.996065060" watchObservedRunningTime="2026-07-24 12:20:38.711662623 +0000 UTC m=+94.997208307" Jul 24 12:20:45.185306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:45.185272 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:20:54.603743 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:20:54.603701 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ln2m5" Jul 24 12:21:10.205762 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:10.205717 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-d84d8bc9-hxlmh" podUID="4e65d668-afad-42b8-b767-1ee7baa2d269" containerName="console" containerID="cri-o://ff34ddad7e507d268d71c33c016a2ecafd4f95272b1451e74ff5e98815464c80" gracePeriod=15 Jul 24 12:21:10.797156 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:10.797124 2578 generic.go:356] "Generic (PLEG): container finished" podID="4e65d668-afad-42b8-b767-1ee7baa2d269" containerID="ff34ddad7e507d268d71c33c016a2ecafd4f95272b1451e74ff5e98815464c80" exitCode=0 Jul 24 12:21:10.797350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:10.797215 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d84d8bc9-hxlmh" event={"ID":"4e65d668-afad-42b8-b767-1ee7baa2d269","Type":"ContainerDied","Data":"ff34ddad7e507d268d71c33c016a2ecafd4f95272b1451e74ff5e98815464c80"} Jul 24 12:21:10.997671 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:10.997640 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:21:11.024767 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024737 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srlsb\" (UniqueName: \"kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.024945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024787 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.024945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024879 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.024945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024905 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.024945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024938 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.025158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.024976 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config\") pod \"4e65d668-afad-42b8-b767-1ee7baa2d269\" (UID: \"4e65d668-afad-42b8-b767-1ee7baa2d269\") " Jul 24 12:21:11.025380 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.025347 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:21:11.025585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.025373 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca" (OuterVolumeSpecName: "service-ca") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:21:11.025585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.025399 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config" (OuterVolumeSpecName: "console-config") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:21:11.027649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.027621 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:21:11.027851 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.027830 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb" (OuterVolumeSpecName: "kube-api-access-srlsb") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "kube-api-access-srlsb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:21:11.027900 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.027844 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4e65d668-afad-42b8-b767-1ee7baa2d269" (UID: "4e65d668-afad-42b8-b767-1ee7baa2d269"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126108 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-oauth-serving-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126133 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-oauth-config\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126143 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4e65d668-afad-42b8-b767-1ee7baa2d269-console-serving-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126151 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-service-ca\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126160 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4e65d668-afad-42b8-b767-1ee7baa2d269-console-config\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.126183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.126169 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srlsb\" (UniqueName: \"kubernetes.io/projected/4e65d668-afad-42b8-b767-1ee7baa2d269-kube-api-access-srlsb\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:21:11.802136 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.802097 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-d84d8bc9-hxlmh" event={"ID":"4e65d668-afad-42b8-b767-1ee7baa2d269","Type":"ContainerDied","Data":"60f35c4f002cb3879be8d2a88296132be2435eb2e2b6decceef06de7eebc8a0f"} Jul 24 12:21:11.802136 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.802127 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-d84d8bc9-hxlmh" Jul 24 12:21:11.802616 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.802167 2578 scope.go:117] "RemoveContainer" containerID="ff34ddad7e507d268d71c33c016a2ecafd4f95272b1451e74ff5e98815464c80" Jul 24 12:21:11.836960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.836918 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:21:11.839078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:11.839047 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-d84d8bc9-hxlmh"] Jul 24 12:21:12.147300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:12.147210 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e65d668-afad-42b8-b767-1ee7baa2d269" path="/var/lib/kubelet/pods/4e65d668-afad-42b8-b767-1ee7baa2d269/volumes" Jul 24 12:21:31.674026 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.673989 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674234 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerName="registry" Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674246 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerName="registry" Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674257 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e65d668-afad-42b8-b767-1ee7baa2d269" containerName="console" Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674262 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e65d668-afad-42b8-b767-1ee7baa2d269" containerName="console" Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674301 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e65d668-afad-42b8-b767-1ee7baa2d269" containerName="console" Jul 24 12:21:31.674483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.674307 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c58a1211-6ad4-4b3a-a037-de7bef79913b" containerName="registry" Jul 24 12:21:31.675953 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.675936 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.679795 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.679772 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 24 12:21:31.680553 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.680526 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 24 12:21:31.680676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.680564 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 24 12:21:31.680676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.680537 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 24 12:21:31.680676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.680533 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-l6rpx\"" Jul 24 12:21:31.680676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.680568 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 24 12:21:31.685245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.685149 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 24 12:21:31.686657 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.686639 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:21:31.772558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772524 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772568 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772597 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772642 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772683 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772704 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xgqdc\" (UniqueName: \"kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.772789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.772725 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.873883 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.873883 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873882 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.873883 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873906 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874231 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873924 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874231 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873946 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874231 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xgqdc\" (UniqueName: \"kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874231 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.873988 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874745 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.874721 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.874740 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.874829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.874759 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.875167 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.875145 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.876600 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.876579 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.876701 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.876684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.882829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.882794 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xgqdc\" (UniqueName: \"kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc\") pod \"console-67b96579f6-52grh\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:31.987006 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:31.986906 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:32.118862 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:32.118826 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:21:32.121851 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:21:32.121822 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31e979ae_f0db_43e2_89eb_9afda947dc19.slice/crio-1853f949f1483fe991431d7e7d68a834d6e4728e57de5f4f46b3b1ee2f9eacab WatchSource:0}: Error finding container 1853f949f1483fe991431d7e7d68a834d6e4728e57de5f4f46b3b1ee2f9eacab: Status 404 returned error can't find the container with id 1853f949f1483fe991431d7e7d68a834d6e4728e57de5f4f46b3b1ee2f9eacab Jul 24 12:21:32.885221 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:32.885166 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b96579f6-52grh" event={"ID":"31e979ae-f0db-43e2-89eb-9afda947dc19","Type":"ContainerStarted","Data":"150131a0aed1515b182666d4fc024953618a1cd171091a424240754465753b12"} Jul 24 12:21:32.885571 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:32.885236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b96579f6-52grh" event={"ID":"31e979ae-f0db-43e2-89eb-9afda947dc19","Type":"ContainerStarted","Data":"1853f949f1483fe991431d7e7d68a834d6e4728e57de5f4f46b3b1ee2f9eacab"} Jul 24 12:21:32.901401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:32.901351 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67b96579f6-52grh" podStartSLOduration=1.901337149 podStartE2EDuration="1.901337149s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:21:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:21:32.900832315 +0000 UTC m=+149.186378012" watchObservedRunningTime="2026-07-24 12:21:32.901337149 +0000 UTC m=+149.186882833" Jul 24 12:21:41.987999 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:41.987955 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:41.988507 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:41.988042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:41.992767 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:41.992740 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:21:42.924483 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:21:42.924456 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:22:35.593393 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.593353 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld"] Jul 24 12:22:35.596422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.596406 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.599292 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.599273 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wzt42\"" Jul 24 12:22:35.599820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.599804 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 12:22:35.599903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.599851 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 12:22:35.607507 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.607485 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld"] Jul 24 12:22:35.648150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.648114 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.648150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.648150 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.648381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.648177 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qh6jn\" (UniqueName: \"kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.748760 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.748724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.748760 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.748761 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.748942 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.748789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qh6jn\" (UniqueName: \"kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.749257 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.749185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.749257 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.749187 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.757145 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.757112 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qh6jn\" (UniqueName: \"kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:35.905545 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:35.905459 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:36.036117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:36.036094 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld"] Jul 24 12:22:36.038898 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:22:36.038862 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11d769ce_0f8b_400d_849e_0e03e4033cf1.slice/crio-2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5 WatchSource:0}: Error finding container 2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5: Status 404 returned error can't find the container with id 2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5 Jul 24 12:22:36.089796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:36.089765 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" event={"ID":"11d769ce-0f8b-400d-849e-0e03e4033cf1","Type":"ContainerStarted","Data":"2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5"} Jul 24 12:22:43.114964 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:43.114923 2578 generic.go:356] "Generic (PLEG): container finished" podID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerID="23a7bfec0615b2a98f2e2e3512bd55ab89429e97eddb188ae0a3380fc4461c0e" exitCode=0 Jul 24 12:22:43.115406 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:43.115015 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" event={"ID":"11d769ce-0f8b-400d-849e-0e03e4033cf1","Type":"ContainerDied","Data":"23a7bfec0615b2a98f2e2e3512bd55ab89429e97eddb188ae0a3380fc4461c0e"} Jul 24 12:22:46.126467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:46.126430 2578 generic.go:356] "Generic (PLEG): container finished" podID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerID="6b34acc8c3eba84d60d6cdb7c58be8866f0e8bdd70761ae2f8e20e241a6a6920" exitCode=0 Jul 24 12:22:46.126924 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:46.126500 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" event={"ID":"11d769ce-0f8b-400d-849e-0e03e4033cf1","Type":"ContainerDied","Data":"6b34acc8c3eba84d60d6cdb7c58be8866f0e8bdd70761ae2f8e20e241a6a6920"} Jul 24 12:22:53.153747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:53.153715 2578 generic.go:356] "Generic (PLEG): container finished" podID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerID="e1eea78c23f0750278e7aa131dd8991a93cf1c54312b87395b179f2fe3262341" exitCode=0 Jul 24 12:22:53.154161 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:53.153760 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" event={"ID":"11d769ce-0f8b-400d-849e-0e03e4033cf1","Type":"ContainerDied","Data":"e1eea78c23f0750278e7aa131dd8991a93cf1c54312b87395b179f2fe3262341"} Jul 24 12:22:54.276246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.276207 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:54.394569 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.394536 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util\") pod \"11d769ce-0f8b-400d-849e-0e03e4033cf1\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " Jul 24 12:22:54.394754 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.394590 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle\") pod \"11d769ce-0f8b-400d-849e-0e03e4033cf1\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " Jul 24 12:22:54.394754 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.394617 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qh6jn\" (UniqueName: \"kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn\") pod \"11d769ce-0f8b-400d-849e-0e03e4033cf1\" (UID: \"11d769ce-0f8b-400d-849e-0e03e4033cf1\") " Jul 24 12:22:54.395301 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.395270 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle" (OuterVolumeSpecName: "bundle") pod "11d769ce-0f8b-400d-849e-0e03e4033cf1" (UID: "11d769ce-0f8b-400d-849e-0e03e4033cf1"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:22:54.396967 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.396944 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn" (OuterVolumeSpecName: "kube-api-access-qh6jn") pod "11d769ce-0f8b-400d-849e-0e03e4033cf1" (UID: "11d769ce-0f8b-400d-849e-0e03e4033cf1"). InnerVolumeSpecName "kube-api-access-qh6jn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:22:54.399944 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.399921 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util" (OuterVolumeSpecName: "util") pod "11d769ce-0f8b-400d-849e-0e03e4033cf1" (UID: "11d769ce-0f8b-400d-849e-0e03e4033cf1"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:22:54.495429 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.495332 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:22:54.495429 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.495372 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/11d769ce-0f8b-400d-849e-0e03e4033cf1-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:22:54.495429 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:54.495382 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qh6jn\" (UniqueName: \"kubernetes.io/projected/11d769ce-0f8b-400d-849e-0e03e4033cf1-kube-api-access-qh6jn\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:22:55.164018 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:55.163991 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" Jul 24 12:22:55.164180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:55.163989 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540t92ld" event={"ID":"11d769ce-0f8b-400d-849e-0e03e4033cf1","Type":"ContainerDied","Data":"2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5"} Jul 24 12:22:55.164180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:55.164111 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2fa5dd9bf05141935111f6ae9b8a24023b557b4d86b2d4f0023b467dcf8e50d5" Jul 24 12:22:58.644111 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644067 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l"] Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644431 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="extract" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644450 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="extract" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644481 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="pull" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644490 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="pull" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644500 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="util" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644508 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="util" Jul 24 12:22:58.644634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.644565 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="11d769ce-0f8b-400d-849e-0e03e4033cf1" containerName="extract" Jul 24 12:22:58.647594 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.647575 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.649410 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.649387 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 24 12:22:58.649547 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.649435 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-l4tcl\"" Jul 24 12:22:58.649547 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.649399 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 24 12:22:58.660291 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.660265 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l"] Jul 24 12:22:58.725057 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.725018 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361ec2ec-6d87-4ff1-8591-c64e463c828b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.725265 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.725076 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ck2hm\" (UniqueName: \"kubernetes.io/projected/361ec2ec-6d87-4ff1-8591-c64e463c828b-kube-api-access-ck2hm\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.826092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.826052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361ec2ec-6d87-4ff1-8591-c64e463c828b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.826290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.826103 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ck2hm\" (UniqueName: \"kubernetes.io/projected/361ec2ec-6d87-4ff1-8591-c64e463c828b-kube-api-access-ck2hm\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.826532 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.826507 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/361ec2ec-6d87-4ff1-8591-c64e463c828b-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.845018 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.844991 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ck2hm\" (UniqueName: \"kubernetes.io/projected/361ec2ec-6d87-4ff1-8591-c64e463c828b-kube-api-access-ck2hm\") pod \"cert-manager-operator-controller-manager-55c444bf86-6x86l\" (UID: \"361ec2ec-6d87-4ff1-8591-c64e463c828b\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:58.956856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:58.956763 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" Jul 24 12:22:59.089340 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:59.089308 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l"] Jul 24 12:22:59.092854 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:22:59.092822 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod361ec2ec_6d87_4ff1_8591_c64e463c828b.slice/crio-1d4eb06419bc8ebf7b8f640161f5ef9bcd18a809224ab117f76a541e1a70952b WatchSource:0}: Error finding container 1d4eb06419bc8ebf7b8f640161f5ef9bcd18a809224ab117f76a541e1a70952b: Status 404 returned error can't find the container with id 1d4eb06419bc8ebf7b8f640161f5ef9bcd18a809224ab117f76a541e1a70952b Jul 24 12:22:59.177800 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:22:59.177762 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" event={"ID":"361ec2ec-6d87-4ff1-8591-c64e463c828b","Type":"ContainerStarted","Data":"1d4eb06419bc8ebf7b8f640161f5ef9bcd18a809224ab117f76a541e1a70952b"} Jul 24 12:23:01.186810 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:01.186770 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" event={"ID":"361ec2ec-6d87-4ff1-8591-c64e463c828b","Type":"ContainerStarted","Data":"5ba9a8ff868495f5613c5874932f1cc57437a4867aa17f58873b5aaa8dd89060"} Jul 24 12:23:01.206502 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:01.206453 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-6x86l" podStartSLOduration=1.561743152 podStartE2EDuration="3.206438265s" totalImagesPullingTime="1.644695113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:22:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:23:01.205444969 +0000 UTC m=+237.490990652" watchObservedRunningTime="2026-07-24 12:23:01.206438265 +0000 UTC m=+237.491983949" Jul 24 12:23:02.270736 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.270705 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4"] Jul 24 12:23:02.273975 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.273952 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.278714 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.278680 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wzt42\"" Jul 24 12:23:02.278860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.278721 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 12:23:02.279122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.279103 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 12:23:02.300769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.300742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4"] Jul 24 12:23:02.350381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.350340 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.350538 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.350395 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.350538 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.350473 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpblt\" (UniqueName: \"kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.451091 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.451056 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qpblt\" (UniqueName: \"kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.451300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.451099 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.451300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.451229 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.451455 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.451440 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.451528 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.451510 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.468163 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.468117 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpblt\" (UniqueName: \"kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.582887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.582844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:02.717065 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:02.717029 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4"] Jul 24 12:23:02.719996 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:02.719953 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1323d780_1f18_4941_affd_2161bc967614.slice/crio-c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d WatchSource:0}: Error finding container c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d: Status 404 returned error can't find the container with id c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d Jul 24 12:23:03.200390 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.200351 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323d780-1f18-4941-affd-2161bc967614" containerID="88d1f59f1c5322da68c44feddcc10e900ce08e204e688a510e86a568548582d1" exitCode=0 Jul 24 12:23:03.200563 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.200442 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerDied","Data":"88d1f59f1c5322da68c44feddcc10e900ce08e204e688a510e86a568548582d1"} Jul 24 12:23:03.200563 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.200520 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerStarted","Data":"c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d"} Jul 24 12:23:03.487379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.487348 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rgndn"] Jul 24 12:23:03.490378 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.490362 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.494065 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.494034 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 24 12:23:03.494286 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.494269 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-88ncf\"" Jul 24 12:23:03.494386 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.494285 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 24 12:23:03.503710 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.503682 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rgndn"] Jul 24 12:23:03.560665 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.560626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.560810 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.560690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qlv7\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-kube-api-access-8qlv7\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.661406 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.661364 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.661605 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.661446 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8qlv7\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-kube-api-access-8qlv7\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.669866 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.669839 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.670046 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.670026 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qlv7\" (UniqueName: \"kubernetes.io/projected/dce78e4f-c041-47fc-a946-55801b430cba-kube-api-access-8qlv7\") pod \"cert-manager-webhook-6486ffd997-rgndn\" (UID: \"dce78e4f-c041-47fc-a946-55801b430cba\") " pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.816718 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.816678 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:03.967870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:03.967838 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-rgndn"] Jul 24 12:23:03.971556 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:03.971524 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddce78e4f_c041_47fc_a946_55801b430cba.slice/crio-c5e122039528dd8eb6a9d02036c765252e8d1277a3f10e1116f3e41171002398 WatchSource:0}: Error finding container c5e122039528dd8eb6a9d02036c765252e8d1277a3f10e1116f3e41171002398: Status 404 returned error can't find the container with id c5e122039528dd8eb6a9d02036c765252e8d1277a3f10e1116f3e41171002398 Jul 24 12:23:04.206298 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:04.206217 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" event={"ID":"dce78e4f-c041-47fc-a946-55801b430cba","Type":"ContainerStarted","Data":"c5e122039528dd8eb6a9d02036c765252e8d1277a3f10e1116f3e41171002398"} Jul 24 12:23:05.214460 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:05.214418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerStarted","Data":"8eafa109a70bd75ee4dbd8467b396013552332233588c318906785ae7dd16e78"} Jul 24 12:23:06.220357 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:06.220320 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323d780-1f18-4941-affd-2161bc967614" containerID="8eafa109a70bd75ee4dbd8467b396013552332233588c318906785ae7dd16e78" exitCode=0 Jul 24 12:23:06.220871 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:06.220370 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerDied","Data":"8eafa109a70bd75ee4dbd8467b396013552332233588c318906785ae7dd16e78"} Jul 24 12:23:07.226520 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:07.226433 2578 generic.go:356] "Generic (PLEG): container finished" podID="1323d780-1f18-4941-affd-2161bc967614" containerID="7d8722f618cdf60f0f818624ec17414036f69f83315f8181688bf8fdb73b3e6c" exitCode=0 Jul 24 12:23:07.226908 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:07.226522 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerDied","Data":"7d8722f618cdf60f0f818624ec17414036f69f83315f8181688bf8fdb73b3e6c"} Jul 24 12:23:07.228003 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:07.227978 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" event={"ID":"dce78e4f-c041-47fc-a946-55801b430cba","Type":"ContainerStarted","Data":"fdd18c0c7af94987b3015f5e7f147b9066427615eb49380b022311a63fcb4dfb"} Jul 24 12:23:07.228131 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:07.228031 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:07.282555 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:07.282498 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" podStartSLOduration=1.242659461 podStartE2EDuration="4.282480993s" totalImagesPullingTime="3.039821532s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:23:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:23:07.278392234 +0000 UTC m=+243.563937920" watchObservedRunningTime="2026-07-24 12:23:07.282480993 +0000 UTC m=+243.568026677" Jul 24 12:23:08.353466 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.353441 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:08.398560 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.398528 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle\") pod \"1323d780-1f18-4941-affd-2161bc967614\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " Jul 24 12:23:08.398735 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.398599 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpblt\" (UniqueName: \"kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt\") pod \"1323d780-1f18-4941-affd-2161bc967614\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " Jul 24 12:23:08.398735 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.398655 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util\") pod \"1323d780-1f18-4941-affd-2161bc967614\" (UID: \"1323d780-1f18-4941-affd-2161bc967614\") " Jul 24 12:23:08.399022 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.398993 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle" (OuterVolumeSpecName: "bundle") pod "1323d780-1f18-4941-affd-2161bc967614" (UID: "1323d780-1f18-4941-affd-2161bc967614"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:23:08.400847 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.400812 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt" (OuterVolumeSpecName: "kube-api-access-qpblt") pod "1323d780-1f18-4941-affd-2161bc967614" (UID: "1323d780-1f18-4941-affd-2161bc967614"). InnerVolumeSpecName "kube-api-access-qpblt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:23:08.403049 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.403027 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util" (OuterVolumeSpecName: "util") pod "1323d780-1f18-4941-affd-2161bc967614" (UID: "1323d780-1f18-4941-affd-2161bc967614"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:23:08.499904 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.499809 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpblt\" (UniqueName: \"kubernetes.io/projected/1323d780-1f18-4941-affd-2161bc967614-kube-api-access-qpblt\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:08.499904 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.499841 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:08.499904 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:08.499853 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1323d780-1f18-4941-affd-2161bc967614-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:09.237063 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:09.237026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" event={"ID":"1323d780-1f18-4941-affd-2161bc967614","Type":"ContainerDied","Data":"c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d"} Jul 24 12:23:09.237263 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:09.237066 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmvgv4" Jul 24 12:23:09.237263 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:09.237085 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c1c8bdefbda09645d9e8ff1adefa6d5c1091c762594888eeb7265d493572b86d" Jul 24 12:23:13.234146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:13.234114 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-rgndn" Jul 24 12:23:14.844825 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.844790 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ggs62"] Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845044 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="util" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845054 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="util" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845063 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="extract" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845068 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="extract" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845078 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="pull" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845083 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="pull" Jul 24 12:23:14.845187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.845126 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1323d780-1f18-4941-affd-2161bc967614" containerName="extract" Jul 24 12:23:14.868759 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.868727 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ggs62"] Jul 24 12:23:14.868914 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.868821 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:14.870708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.870677 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-pfpbl\"" Jul 24 12:23:14.948336 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.948301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-bound-sa-token\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:14.948336 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:14.948345 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25lrh\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-kube-api-access-25lrh\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.049362 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.049316 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-bound-sa-token\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.049362 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.049369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25lrh\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-kube-api-access-25lrh\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.058246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.058185 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25lrh\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-kube-api-access-25lrh\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.058421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.058392 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ba92f34d-42bb-48bc-9239-6857f194062c-bound-sa-token\") pod \"cert-manager-55b4d954d5-ggs62\" (UID: \"ba92f34d-42bb-48bc-9239-6857f194062c\") " pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.177799 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.177708 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-ggs62" Jul 24 12:23:15.325448 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:15.325412 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-ggs62"] Jul 24 12:23:15.329185 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:15.329158 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podba92f34d_42bb_48bc_9239_6857f194062c.slice/crio-6b502f45610b954f70537eff5838f16c3ef683b486cef5136146d5cee13642df WatchSource:0}: Error finding container 6b502f45610b954f70537eff5838f16c3ef683b486cef5136146d5cee13642df: Status 404 returned error can't find the container with id 6b502f45610b954f70537eff5838f16c3ef683b486cef5136146d5cee13642df Jul 24 12:23:16.270815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:16.270783 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ggs62" event={"ID":"ba92f34d-42bb-48bc-9239-6857f194062c","Type":"ContainerStarted","Data":"d16cd1b28ac22a8a11724ec80518e00723e06ab1d85f2174d2f5d639a5b3eac5"} Jul 24 12:23:16.270815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:16.270823 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-ggs62" event={"ID":"ba92f34d-42bb-48bc-9239-6857f194062c","Type":"ContainerStarted","Data":"6b502f45610b954f70537eff5838f16c3ef683b486cef5136146d5cee13642df"} Jul 24 12:23:16.288106 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:16.288056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-ggs62" podStartSLOduration=2.288038919 podStartE2EDuration="2.288038919s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:23:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:23:16.287276454 +0000 UTC m=+252.572822155" watchObservedRunningTime="2026-07-24 12:23:16.288038919 +0000 UTC m=+252.573584604" Jul 24 12:23:28.314814 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.314737 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn"] Jul 24 12:23:28.316854 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.316836 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.320252 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.320227 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 24 12:23:28.320529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.320513 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-wzt42\"" Jul 24 12:23:28.320802 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.320786 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 24 12:23:28.326868 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.326845 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn"] Jul 24 12:23:28.451184 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.451143 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.451372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.451251 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.451372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.451287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26nzj\" (UniqueName: \"kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.552559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.552524 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.552751 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.552574 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26nzj\" (UniqueName: \"kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.552751 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.552616 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.552974 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.552950 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.553049 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.552974 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.563159 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.563119 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26nzj\" (UniqueName: \"kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.627014 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.626926 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:28.753825 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:28.753793 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn"] Jul 24 12:23:28.756741 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:28.756716 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b7df73b_9800_4d3e_a5a4_8a3ccd3de082.slice/crio-cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804 WatchSource:0}: Error finding container cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804: Status 404 returned error can't find the container with id cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804 Jul 24 12:23:29.321662 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:29.321627 2578 generic.go:356] "Generic (PLEG): container finished" podID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerID="b056353ec4964710b5959fc61aa2e01f13e4e0bd002a7f26d71088eaf08e30a4" exitCode=0 Jul 24 12:23:29.322041 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:29.321724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" event={"ID":"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082","Type":"ContainerDied","Data":"b056353ec4964710b5959fc61aa2e01f13e4e0bd002a7f26d71088eaf08e30a4"} Jul 24 12:23:29.322041 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:29.321780 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" event={"ID":"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082","Type":"ContainerStarted","Data":"cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804"} Jul 24 12:23:30.326953 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:30.326921 2578 generic.go:356] "Generic (PLEG): container finished" podID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerID="353b23e0ddbfc251e0f78e86b8f6d1051ad49df527ad15f73fa64ef2ebd66034" exitCode=0 Jul 24 12:23:30.327329 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:30.326979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" event={"ID":"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082","Type":"ContainerDied","Data":"353b23e0ddbfc251e0f78e86b8f6d1051ad49df527ad15f73fa64ef2ebd66034"} Jul 24 12:23:31.333002 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:31.332968 2578 generic.go:356] "Generic (PLEG): container finished" podID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerID="f6cdf362e6cf980811994d077e0e2c56151b7f3960c1898307fdc45bb46f7dec" exitCode=0 Jul 24 12:23:31.333352 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:31.333041 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" event={"ID":"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082","Type":"ContainerDied","Data":"f6cdf362e6cf980811994d077e0e2c56151b7f3960c1898307fdc45bb46f7dec"} Jul 24 12:23:32.457724 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.457700 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:32.582995 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.582958 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util\") pod \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " Jul 24 12:23:32.582995 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.583007 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-26nzj\" (UniqueName: \"kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj\") pod \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " Jul 24 12:23:32.583245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.583049 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle\") pod \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\" (UID: \"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082\") " Jul 24 12:23:32.583978 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.583931 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle" (OuterVolumeSpecName: "bundle") pod "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" (UID: "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:23:32.585273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.585246 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj" (OuterVolumeSpecName: "kube-api-access-26nzj") pod "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" (UID: "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082"). InnerVolumeSpecName "kube-api-access-26nzj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:23:32.588816 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.588746 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util" (OuterVolumeSpecName: "util") pod "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" (UID: "5b7df73b-9800-4d3e-a5a4-8a3ccd3de082"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:23:32.683780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.683745 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-26nzj\" (UniqueName: \"kubernetes.io/projected/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-kube-api-access-26nzj\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:32.683780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.683775 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:32.683780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:32.683783 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5b7df73b-9800-4d3e-a5a4-8a3ccd3de082-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:23:33.342418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:33.342375 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" event={"ID":"5b7df73b-9800-4d3e-a5a4-8a3ccd3de082","Type":"ContainerDied","Data":"cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804"} Jul 24 12:23:33.342418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:33.342405 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afpxndn" Jul 24 12:23:33.342418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:33.342426 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cd278ef754f7c0d91e4e58fcf8d36b23a00c2fc7d477d2e0a1ede13d57f15804" Jul 24 12:23:36.979247 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979208 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj"] Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979453 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="extract" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979463 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="extract" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979476 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="util" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979482 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="util" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979489 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="pull" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979494 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="pull" Jul 24 12:23:36.979631 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.979556 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5b7df73b-9800-4d3e-a5a4-8a3ccd3de082" containerName="extract" Jul 24 12:23:36.981322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.981305 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:36.984330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.984310 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 24 12:23:36.984439 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.984343 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ggthh\"" Jul 24 12:23:36.985000 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.984985 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 24 12:23:36.985305 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.985288 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 24 12:23:36.985723 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.985704 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 24 12:23:36.985868 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:36.985852 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 24 12:23:37.001349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.001327 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj"] Jul 24 12:23:37.117413 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.117373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-metrics-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.117611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.117428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.117611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.117522 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a5d917e4-7933-4b9b-b63a-5219edd81908-manager-config\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.117722 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.117615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtv96\" (UniqueName: \"kubernetes.io/projected/a5d917e4-7933-4b9b-b63a-5219edd81908-kube-api-access-xtv96\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.218474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.218436 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtv96\" (UniqueName: \"kubernetes.io/projected/a5d917e4-7933-4b9b-b63a-5219edd81908-kube-api-access-xtv96\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.218474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.218485 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-metrics-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.218717 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.218609 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.218717 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.218684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a5d917e4-7933-4b9b-b63a-5219edd81908-manager-config\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.219353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.219322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/a5d917e4-7933-4b9b-b63a-5219edd81908-manager-config\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.221069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.221047 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.221222 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.221187 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/a5d917e4-7933-4b9b-b63a-5219edd81908-metrics-cert\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.227918 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.227891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtv96\" (UniqueName: \"kubernetes.io/projected/a5d917e4-7933-4b9b-b63a-5219edd81908-kube-api-access-xtv96\") pod \"lws-controller-manager-d4b69f9c5-52nlj\" (UID: \"a5d917e4-7933-4b9b-b63a-5219edd81908\") " pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.290045 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.290008 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:37.426988 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:37.426953 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj"] Jul 24 12:23:37.430110 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:37.430082 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5d917e4_7933_4b9b_b63a_5219edd81908.slice/crio-325ce3e0271f929408718375f89406c621999202cd3a516cf711cf0fe5af75db WatchSource:0}: Error finding container 325ce3e0271f929408718375f89406c621999202cd3a516cf711cf0fe5af75db: Status 404 returned error can't find the container with id 325ce3e0271f929408718375f89406c621999202cd3a516cf711cf0fe5af75db Jul 24 12:23:38.369801 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:38.369756 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" event={"ID":"a5d917e4-7933-4b9b-b63a-5219edd81908","Type":"ContainerStarted","Data":"325ce3e0271f929408718375f89406c621999202cd3a516cf711cf0fe5af75db"} Jul 24 12:23:39.374927 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:39.374824 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" event={"ID":"a5d917e4-7933-4b9b-b63a-5219edd81908","Type":"ContainerStarted","Data":"cb77e2bb059994b545e03355bcca532df2b811962169edbebab4e9d154160dac"} Jul 24 12:23:39.375355 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:39.374937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:39.400991 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:39.400929 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" podStartSLOduration=1.74860094 podStartE2EDuration="3.40091052s" totalImagesPullingTime="1.65230958s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:23:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:23:39.399506761 +0000 UTC m=+275.685052447" watchObservedRunningTime="2026-07-24 12:23:39.40091052 +0000 UTC m=+275.686456206" Jul 24 12:23:46.366931 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.366892 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96"] Jul 24 12:23:46.369067 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.369045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.373423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.373403 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 24 12:23:46.374064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.374044 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 24 12:23:46.379119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.379104 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 24 12:23:46.379215 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.379133 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 24 12:23:46.379507 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.379489 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-w2xqh\"" Jul 24 12:23:46.408025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.407996 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96"] Jul 24 12:23:46.497896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.497855 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-manifests\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.498103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.497906 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gn48p\" (UniqueName: \"kubernetes.io/projected/053ad4b0-bb32-41aa-86ec-375a95743f7d-kube-api-access-gn48p\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.498103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.497927 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-tmp\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.498103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.498017 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.498103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.498049 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.598825 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.598789 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.598825 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.598829 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-manifests\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.599058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.598862 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gn48p\" (UniqueName: \"kubernetes.io/projected/053ad4b0-bb32-41aa-86ec-375a95743f7d-kube-api-access-gn48p\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.599058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.598880 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-tmp\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.599058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.598913 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.599390 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.599358 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-manifests\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.601296 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.601267 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/053ad4b0-bb32-41aa-86ec-375a95743f7d-tmp\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.601552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.601533 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.601613 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.601575 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/053ad4b0-bb32-41aa-86ec-375a95743f7d-webhook-cert\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.610008 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.609984 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gn48p\" (UniqueName: \"kubernetes.io/projected/053ad4b0-bb32-41aa-86ec-375a95743f7d-kube-api-access-gn48p\") pod \"opendatahub-operator-controller-manager-565c4d787c-brg96\" (UID: \"053ad4b0-bb32-41aa-86ec-375a95743f7d\") " pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.679003 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.678905 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:46.838654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:46.838619 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96"] Jul 24 12:23:46.842221 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:23:46.842159 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod053ad4b0_bb32_41aa_86ec_375a95743f7d.slice/crio-b506c04b7b19c80a6aac7398f3bd93667664b128eb90a5132a992eacb7e8b706 WatchSource:0}: Error finding container b506c04b7b19c80a6aac7398f3bd93667664b128eb90a5132a992eacb7e8b706: Status 404 returned error can't find the container with id b506c04b7b19c80a6aac7398f3bd93667664b128eb90a5132a992eacb7e8b706 Jul 24 12:23:47.407497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:47.407457 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" event={"ID":"053ad4b0-bb32-41aa-86ec-375a95743f7d","Type":"ContainerStarted","Data":"b506c04b7b19c80a6aac7398f3bd93667664b128eb90a5132a992eacb7e8b706"} Jul 24 12:23:50.381171 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:50.381140 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-d4b69f9c5-52nlj" Jul 24 12:23:50.423451 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:50.423411 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" event={"ID":"053ad4b0-bb32-41aa-86ec-375a95743f7d","Type":"ContainerStarted","Data":"6c06697ea5f87174c35c2915f6aa2eab9e1f465137b8fd69a6894a14c55b00b4"} Jul 24 12:23:51.428947 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:51.428906 2578 generic.go:356] "Generic (PLEG): container finished" podID="053ad4b0-bb32-41aa-86ec-375a95743f7d" containerID="6c06697ea5f87174c35c2915f6aa2eab9e1f465137b8fd69a6894a14c55b00b4" exitCode=0 Jul 24 12:23:51.429383 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:51.428995 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" event={"ID":"053ad4b0-bb32-41aa-86ec-375a95743f7d","Type":"ContainerDied","Data":"6c06697ea5f87174c35c2915f6aa2eab9e1f465137b8fd69a6894a14c55b00b4"} Jul 24 12:23:52.434888 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:52.434848 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" event={"ID":"053ad4b0-bb32-41aa-86ec-375a95743f7d","Type":"ContainerStarted","Data":"a579197c6bd4f1fdbec65a925b3b44415f80522cb4e6ec63caef40c8a08d7b18"} Jul 24 12:23:52.435340 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:52.434952 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:23:52.466597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:23:52.466541 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" podStartSLOduration=2.132713581 podStartE2EDuration="6.466513897s" totalImagesPullingTime="3.281512282s" totalInitContainerRuntime="1.052288034s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:23:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:23:52.465417416 +0000 UTC m=+288.750963149" watchObservedRunningTime="2026-07-24 12:23:52.466513897 +0000 UTC m=+288.752059573" Jul 24 12:24:03.441970 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:03.441937 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-565c4d787c-brg96" Jul 24 12:24:04.039549 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:04.039520 2578 kubelet.go:1648] "Image garbage collection succeeded" Jul 24 12:24:28.750266 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.750227 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf"] Jul 24 12:24:28.753658 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.753641 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.756589 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.756539 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 24 12:24:28.756726 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.756651 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 24 12:24:28.757567 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.757548 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zn2qk\"" Jul 24 12:24:28.757683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.757629 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 24 12:24:28.771270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.771244 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf"] Jul 24 12:24:28.819695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819655 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.819879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819709 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.819879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819740 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.819879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.819879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819826 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.819879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819877 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.820076 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.820076 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819931 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6h27\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-kube-api-access-w6h27\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.820076 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.819952 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.920624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920580 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.920784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.920784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920664 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.920784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6h27\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-kube-api-access-w6h27\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.920784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920743 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.920888 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.921054 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.921188 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921326 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.921258 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.921353 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.921880 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.921857 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.923183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.923158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.923513 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.923494 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.932264 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.932234 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6h27\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-kube-api-access-w6h27\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:28.932568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:28.932546 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/19a0e2b6-d80e-45c7-8527-b1e010a76eb7-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf\" (UID: \"19a0e2b6-d80e-45c7-8527-b1e010a76eb7\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:29.063727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:29.063681 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:29.200813 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:29.200786 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf"] Jul 24 12:24:29.203253 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:24:29.203225 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod19a0e2b6_d80e_45c7_8527_b1e010a76eb7.slice/crio-78b7dc579bd5185affdcec7bebdc2ed076c480e9a71392ea1e917ae83126e669 WatchSource:0}: Error finding container 78b7dc579bd5185affdcec7bebdc2ed076c480e9a71392ea1e917ae83126e669: Status 404 returned error can't find the container with id 78b7dc579bd5185affdcec7bebdc2ed076c480e9a71392ea1e917ae83126e669 Jul 24 12:24:29.205155 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:29.205140 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 12:24:29.589414 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:29.589378 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" event={"ID":"19a0e2b6-d80e-45c7-8527-b1e010a76eb7","Type":"ContainerStarted","Data":"78b7dc579bd5185affdcec7bebdc2ed076c480e9a71392ea1e917ae83126e669"} Jul 24 12:24:31.911150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:31.907049 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:24:31.911150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:31.907238 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:24:31.911150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:31.907325 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:24:32.603943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:32.603908 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" event={"ID":"19a0e2b6-d80e-45c7-8527-b1e010a76eb7","Type":"ContainerStarted","Data":"c7ede20373ac4da3f383cc271e6e54aeae953c5e242d727140d6fc4f06d1107e"} Jul 24 12:24:32.628715 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:32.628657 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" podStartSLOduration=1.927596856 podStartE2EDuration="4.628640619s" totalImagesPullingTime="2.701043763s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:24:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:24:32.625446992 +0000 UTC m=+328.910992702" watchObservedRunningTime="2026-07-24 12:24:32.628640619 +0000 UTC m=+328.914186303" Jul 24 12:24:33.064899 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:33.064861 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:33.069589 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:33.069565 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:33.608103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:33.608067 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:24:33.608982 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:24:33.608963 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf" Jul 24 12:25:08.313245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.313138 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:08.318372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.318347 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:08.323473 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.323445 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-lq6mt\"" Jul 24 12:25:08.323624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.323591 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 24 12:25:08.323915 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.323899 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 24 12:25:08.335651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.335624 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:08.354325 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.354286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vkfj\" (UniqueName: \"kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj\") pod \"kuadrant-operator-catalog-wqgms\" (UID: \"dd1ddcae-44ea-4159-8324-81609dc7ab9a\") " pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:08.454892 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.454856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vkfj\" (UniqueName: \"kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj\") pod \"kuadrant-operator-catalog-wqgms\" (UID: \"dd1ddcae-44ea-4159-8324-81609dc7ab9a\") " pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:08.469154 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.469116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vkfj\" (UniqueName: \"kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj\") pod \"kuadrant-operator-catalog-wqgms\" (UID: \"dd1ddcae-44ea-4159-8324-81609dc7ab9a\") " pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:08.578752 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.578666 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:08.578947 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.578932 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:08.714339 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.714304 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:08.717274 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:08.717246 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd1ddcae_44ea_4159_8324_81609dc7ab9a.slice/crio-61c2f45555bea1b4cbbd5d5f94b31757975f59abcce09b167a390dbabe985327 WatchSource:0}: Error finding container 61c2f45555bea1b4cbbd5d5f94b31757975f59abcce09b167a390dbabe985327: Status 404 returned error can't find the container with id 61c2f45555bea1b4cbbd5d5f94b31757975f59abcce09b167a390dbabe985327 Jul 24 12:25:08.755614 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.755575 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" event={"ID":"dd1ddcae-44ea-4159-8324-81609dc7ab9a","Type":"ContainerStarted","Data":"61c2f45555bea1b4cbbd5d5f94b31757975f59abcce09b167a390dbabe985327"} Jul 24 12:25:08.789789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.789754 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vn6bp"] Jul 24 12:25:08.792450 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.792432 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:08.804625 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.804595 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vn6bp"] Jul 24 12:25:08.858220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.858106 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47r5z\" (UniqueName: \"kubernetes.io/projected/eec1bc19-9682-4c58-9e2c-6e09a11114b8-kube-api-access-47r5z\") pod \"kuadrant-operator-catalog-vn6bp\" (UID: \"eec1bc19-9682-4c58-9e2c-6e09a11114b8\") " pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:08.959089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.959047 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47r5z\" (UniqueName: \"kubernetes.io/projected/eec1bc19-9682-4c58-9e2c-6e09a11114b8-kube-api-access-47r5z\") pod \"kuadrant-operator-catalog-vn6bp\" (UID: \"eec1bc19-9682-4c58-9e2c-6e09a11114b8\") " pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:08.978280 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:08.978245 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47r5z\" (UniqueName: \"kubernetes.io/projected/eec1bc19-9682-4c58-9e2c-6e09a11114b8-kube-api-access-47r5z\") pod \"kuadrant-operator-catalog-vn6bp\" (UID: \"eec1bc19-9682-4c58-9e2c-6e09a11114b8\") " pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:09.102309 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:09.102268 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:09.256421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:09.256386 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-vn6bp"] Jul 24 12:25:09.259714 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:09.259688 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeec1bc19_9682_4c58_9e2c_6e09a11114b8.slice/crio-13ec315f6b740e7c7588b07282c6525438658003d212d956993067cea2999bd6 WatchSource:0}: Error finding container 13ec315f6b740e7c7588b07282c6525438658003d212d956993067cea2999bd6: Status 404 returned error can't find the container with id 13ec315f6b740e7c7588b07282c6525438658003d212d956993067cea2999bd6 Jul 24 12:25:09.760510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:09.760465 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" event={"ID":"eec1bc19-9682-4c58-9e2c-6e09a11114b8","Type":"ContainerStarted","Data":"13ec315f6b740e7c7588b07282c6525438658003d212d956993067cea2999bd6"} Jul 24 12:25:11.772782 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:11.772738 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" event={"ID":"dd1ddcae-44ea-4159-8324-81609dc7ab9a","Type":"ContainerStarted","Data":"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6"} Jul 24 12:25:11.773262 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:11.772806 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" podUID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" containerName="registry-server" containerID="cri-o://f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6" gracePeriod=2 Jul 24 12:25:11.774274 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:11.774246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" event={"ID":"eec1bc19-9682-4c58-9e2c-6e09a11114b8","Type":"ContainerStarted","Data":"6936d0d9ebfba39ac1029aa3ac1f045dbbe876e997f3c4769c1e749985f48027"} Jul 24 12:25:11.799886 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:11.799839 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" podStartSLOduration=1.435645495 podStartE2EDuration="3.799822336s" totalImagesPullingTime="2.364176841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:11.796977959 +0000 UTC m=+368.082523643" watchObservedRunningTime="2026-07-24 12:25:11.799822336 +0000 UTC m=+368.085368063" Jul 24 12:25:11.828106 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:11.828049 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" podStartSLOduration=2.003741293 podStartE2EDuration="3.828035028s" totalImagesPullingTime="1.824293735s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:11.825892713 +0000 UTC m=+368.111438417" watchObservedRunningTime="2026-07-24 12:25:11.828035028 +0000 UTC m=+368.113580765" Jul 24 12:25:12.017842 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.017814 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:12.089283 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.089159 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vkfj\" (UniqueName: \"kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj\") pod \"dd1ddcae-44ea-4159-8324-81609dc7ab9a\" (UID: \"dd1ddcae-44ea-4159-8324-81609dc7ab9a\") " Jul 24 12:25:12.091462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.091439 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj" (OuterVolumeSpecName: "kube-api-access-4vkfj") pod "dd1ddcae-44ea-4159-8324-81609dc7ab9a" (UID: "dd1ddcae-44ea-4159-8324-81609dc7ab9a"). InnerVolumeSpecName "kube-api-access-4vkfj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:25:12.190094 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.190054 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vkfj\" (UniqueName: \"kubernetes.io/projected/dd1ddcae-44ea-4159-8324-81609dc7ab9a-kube-api-access-4vkfj\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:12.779802 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.779769 2578 generic.go:356] "Generic (PLEG): container finished" podID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" containerID="f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6" exitCode=0 Jul 24 12:25:12.780295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.779810 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" Jul 24 12:25:12.780295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.779858 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" event={"ID":"dd1ddcae-44ea-4159-8324-81609dc7ab9a","Type":"ContainerDied","Data":"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6"} Jul 24 12:25:12.780295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.779915 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-wqgms" event={"ID":"dd1ddcae-44ea-4159-8324-81609dc7ab9a","Type":"ContainerDied","Data":"61c2f45555bea1b4cbbd5d5f94b31757975f59abcce09b167a390dbabe985327"} Jul 24 12:25:12.780295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.779945 2578 scope.go:117] "RemoveContainer" containerID="f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6" Jul 24 12:25:12.788601 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.788581 2578 scope.go:117] "RemoveContainer" containerID="f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6" Jul 24 12:25:12.788860 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:25:12.788842 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6\": container with ID starting with f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6 not found: ID does not exist" containerID="f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6" Jul 24 12:25:12.788905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.788870 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6"} err="failed to get container status \"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6\": rpc error: code = NotFound desc = could not find container \"f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6\": container with ID starting with f88a8ed6a2c0a23a212d6bf469881e0757af26f0c547c62aa598b7bc67ff8eb6 not found: ID does not exist" Jul 24 12:25:12.806858 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.806832 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:12.817279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:12.817250 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-wqgms"] Jul 24 12:25:14.148117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:14.148087 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" path="/var/lib/kubelet/pods/dd1ddcae-44ea-4159-8324-81609dc7ab9a/volumes" Jul 24 12:25:19.102934 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:19.102889 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:19.103388 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:19.102967 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:19.124336 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:19.124301 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:19.830594 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:19.830569 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-vn6bp" Jul 24 12:25:23.820731 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.820695 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn"] Jul 24 12:25:23.821162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.821130 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" containerName="registry-server" Jul 24 12:25:23.821162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.821148 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" containerName="registry-server" Jul 24 12:25:23.821309 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.821252 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="dd1ddcae-44ea-4159-8324-81609dc7ab9a" containerName="registry-server" Jul 24 12:25:23.825441 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.825420 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.827841 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.827817 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nl2w7\"" Jul 24 12:25:23.837858 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.837835 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn"] Jul 24 12:25:23.884246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.884179 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.884246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.884250 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whsk7\" (UniqueName: \"kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.884469 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.884370 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.984756 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.984713 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.984952 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.984772 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.984952 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.984801 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whsk7\" (UniqueName: \"kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.985181 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.985158 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.985307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.985180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:23.995258 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:23.995233 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whsk7\" (UniqueName: \"kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:24.134279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.134177 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:24.220211 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.220148 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m"] Jul 24 12:25:24.224923 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.224895 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.234157 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.234087 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m"] Jul 24 12:25:24.280646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.280620 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn"] Jul 24 12:25:24.283246 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:24.283217 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98ae81f5_4281_4ba0_b422_96b5540f5744.slice/crio-3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412 WatchSource:0}: Error finding container 3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412: Status 404 returned error can't find the container with id 3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412 Jul 24 12:25:24.287250 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.287229 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.287342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.287259 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7g4p\" (UniqueName: \"kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.287342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.287287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.388303 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.388220 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.388303 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.388277 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7g4p\" (UniqueName: \"kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.388499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.388313 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.388650 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.388628 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.388706 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.388633 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.397242 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.397219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7g4p\" (UniqueName: \"kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.538258 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.538227 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:24.620283 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.620249 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz"] Jul 24 12:25:24.625586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.625562 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.635323 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.635290 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz"] Jul 24 12:25:24.676396 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.676371 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m"] Jul 24 12:25:24.678008 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:24.677979 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c2a3a95_b9a8_476e_b0dc_00b91c8c1d47.slice/crio-c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809 WatchSource:0}: Error finding container c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809: Status 404 returned error can't find the container with id c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809 Jul 24 12:25:24.691490 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.691467 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.691558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.691506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqbsp\" (UniqueName: \"kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.691558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.691532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.792217 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.792167 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.792372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.792242 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqbsp\" (UniqueName: \"kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.792372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.792264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.792565 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.792544 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.792630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.792549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.801498 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.801469 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqbsp\" (UniqueName: \"kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:24.832405 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.832366 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerID="80f8e4205bd42e175db287fdb51048fff896d9c35f06a88da638204acf706b56" exitCode=0 Jul 24 12:25:24.832803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.832461 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" event={"ID":"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47","Type":"ContainerDied","Data":"80f8e4205bd42e175db287fdb51048fff896d9c35f06a88da638204acf706b56"} Jul 24 12:25:24.832803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.832525 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" event={"ID":"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47","Type":"ContainerStarted","Data":"c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809"} Jul 24 12:25:24.834253 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.834231 2578 generic.go:356] "Generic (PLEG): container finished" podID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerID="399c320ca38d836bf24330aba7b16f314963a482811e1b517697f2bfe376c724" exitCode=0 Jul 24 12:25:24.834363 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.834263 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" event={"ID":"98ae81f5-4281-4ba0-b422-96b5540f5744","Type":"ContainerDied","Data":"399c320ca38d836bf24330aba7b16f314963a482811e1b517697f2bfe376c724"} Jul 24 12:25:24.834363 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.834317 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" event={"ID":"98ae81f5-4281-4ba0-b422-96b5540f5744","Type":"ContainerStarted","Data":"3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412"} Jul 24 12:25:24.936306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:24.936210 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:25.046598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.046565 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc"] Jul 24 12:25:25.050576 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.050547 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.070937 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.070873 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc"] Jul 24 12:25:25.087880 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.087856 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz"] Jul 24 12:25:25.089975 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:25.089948 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192d104b_c5b9_46ba_8b7f_1e20cfa18300.slice/crio-2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2 WatchSource:0}: Error finding container 2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2: Status 404 returned error can't find the container with id 2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2 Jul 24 12:25:25.094777 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.094749 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.094911 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.094863 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tf2rm\" (UniqueName: \"kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.094911 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.094897 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.195571 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.195539 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tf2rm\" (UniqueName: \"kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.195673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.195590 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.195673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.195638 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.195993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.195973 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.196048 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.196010 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.206396 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.206359 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tf2rm\" (UniqueName: \"kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.362971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.362930 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:25.505024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.505001 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc"] Jul 24 12:25:25.507095 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:25.507073 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod201d23be_aaf9_4abe_a911_b4ec3e37ed0d.slice/crio-3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24 WatchSource:0}: Error finding container 3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24: Status 404 returned error can't find the container with id 3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24 Jul 24 12:25:25.840701 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.840672 2578 generic.go:356] "Generic (PLEG): container finished" podID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerID="cc31caeb38c19815e3b796acbf65fc0982ef326442ab611d090e743501c39a54" exitCode=0 Jul 24 12:25:25.841060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.840751 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" event={"ID":"192d104b-c5b9-46ba-8b7f-1e20cfa18300","Type":"ContainerDied","Data":"cc31caeb38c19815e3b796acbf65fc0982ef326442ab611d090e743501c39a54"} Jul 24 12:25:25.841060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.840813 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" event={"ID":"192d104b-c5b9-46ba-8b7f-1e20cfa18300","Type":"ContainerStarted","Data":"2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2"} Jul 24 12:25:25.842743 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.842712 2578 generic.go:356] "Generic (PLEG): container finished" podID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerID="5ac55c9084bc172421a99741b0c60d92581398b46786c81f08687f68735fb331" exitCode=0 Jul 24 12:25:25.842849 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.842793 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" event={"ID":"98ae81f5-4281-4ba0-b422-96b5540f5744","Type":"ContainerDied","Data":"5ac55c9084bc172421a99741b0c60d92581398b46786c81f08687f68735fb331"} Jul 24 12:25:25.844619 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.844578 2578 generic.go:356] "Generic (PLEG): container finished" podID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerID="da730a36ed768216d4e5a2160ef866c455b0644aa8d1a1cd950d80235e38cdb8" exitCode=0 Jul 24 12:25:25.844684 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.844620 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" event={"ID":"201d23be-aaf9-4abe-a911-b4ec3e37ed0d","Type":"ContainerDied","Data":"da730a36ed768216d4e5a2160ef866c455b0644aa8d1a1cd950d80235e38cdb8"} Jul 24 12:25:25.844684 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:25.844653 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" event={"ID":"201d23be-aaf9-4abe-a911-b4ec3e37ed0d","Type":"ContainerStarted","Data":"3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24"} Jul 24 12:25:26.851415 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.851382 2578 generic.go:356] "Generic (PLEG): container finished" podID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerID="bcad55cedd3fb5dab80ab666b8fdfff8b2663029e67932c7b5b13a038852699e" exitCode=0 Jul 24 12:25:26.851819 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.851468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" event={"ID":"201d23be-aaf9-4abe-a911-b4ec3e37ed0d","Type":"ContainerDied","Data":"bcad55cedd3fb5dab80ab666b8fdfff8b2663029e67932c7b5b13a038852699e"} Jul 24 12:25:26.853350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.853329 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerID="1d93e62253ad1454268c825f9682e46472656dc00b5e3e0ff5c877132521bc0b" exitCode=0 Jul 24 12:25:26.853420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.853406 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" event={"ID":"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47","Type":"ContainerDied","Data":"1d93e62253ad1454268c825f9682e46472656dc00b5e3e0ff5c877132521bc0b"} Jul 24 12:25:26.855472 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.855454 2578 generic.go:356] "Generic (PLEG): container finished" podID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerID="982672c99378bf16344a3b4e2bc51adf253d995fb18f541d18bbfac721b15517" exitCode=0 Jul 24 12:25:26.855560 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.855517 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" event={"ID":"192d104b-c5b9-46ba-8b7f-1e20cfa18300","Type":"ContainerDied","Data":"982672c99378bf16344a3b4e2bc51adf253d995fb18f541d18bbfac721b15517"} Jul 24 12:25:26.857789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.857769 2578 generic.go:356] "Generic (PLEG): container finished" podID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerID="8b77215664f26a24bb7573796e630591f9961d2a0762f174fc0d557b0ec0adcf" exitCode=0 Jul 24 12:25:26.857885 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:26.857808 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" event={"ID":"98ae81f5-4281-4ba0-b422-96b5540f5744","Type":"ContainerDied","Data":"8b77215664f26a24bb7573796e630591f9961d2a0762f174fc0d557b0ec0adcf"} Jul 24 12:25:27.864358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.864324 2578 generic.go:356] "Generic (PLEG): container finished" podID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerID="db43e2ef13d29b1207edfff599a39c318c7b5c955c3d6bdf4dd8c5ed8e04c7e8" exitCode=0 Jul 24 12:25:27.864775 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.864418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" event={"ID":"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47","Type":"ContainerDied","Data":"db43e2ef13d29b1207edfff599a39c318c7b5c955c3d6bdf4dd8c5ed8e04c7e8"} Jul 24 12:25:27.866444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.866419 2578 generic.go:356] "Generic (PLEG): container finished" podID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerID="d8cb23aa081c4cff7e3c57b3425e90411b1f2776a8548cfdec6a97df87e03433" exitCode=0 Jul 24 12:25:27.866541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.866509 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" event={"ID":"192d104b-c5b9-46ba-8b7f-1e20cfa18300","Type":"ContainerDied","Data":"d8cb23aa081c4cff7e3c57b3425e90411b1f2776a8548cfdec6a97df87e03433"} Jul 24 12:25:27.868486 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.868465 2578 generic.go:356] "Generic (PLEG): container finished" podID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerID="e5b135bfa065f0beb1ba4cc5dc3f04ce169ae877dc525ab27ce83e6352527f19" exitCode=0 Jul 24 12:25:27.868573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.868541 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" event={"ID":"201d23be-aaf9-4abe-a911-b4ec3e37ed0d","Type":"ContainerDied","Data":"e5b135bfa065f0beb1ba4cc5dc3f04ce169ae877dc525ab27ce83e6352527f19"} Jul 24 12:25:27.994730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:27.994702 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:28.119694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.119595 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle\") pod \"98ae81f5-4281-4ba0-b422-96b5540f5744\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " Jul 24 12:25:28.119694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.119657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util\") pod \"98ae81f5-4281-4ba0-b422-96b5540f5744\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " Jul 24 12:25:28.119912 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.119702 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whsk7\" (UniqueName: \"kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7\") pod \"98ae81f5-4281-4ba0-b422-96b5540f5744\" (UID: \"98ae81f5-4281-4ba0-b422-96b5540f5744\") " Jul 24 12:25:28.120228 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.120177 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle" (OuterVolumeSpecName: "bundle") pod "98ae81f5-4281-4ba0-b422-96b5540f5744" (UID: "98ae81f5-4281-4ba0-b422-96b5540f5744"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:28.122017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.121987 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7" (OuterVolumeSpecName: "kube-api-access-whsk7") pod "98ae81f5-4281-4ba0-b422-96b5540f5744" (UID: "98ae81f5-4281-4ba0-b422-96b5540f5744"). InnerVolumeSpecName "kube-api-access-whsk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:25:28.124682 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.124662 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util" (OuterVolumeSpecName: "util") pod "98ae81f5-4281-4ba0-b422-96b5540f5744" (UID: "98ae81f5-4281-4ba0-b422-96b5540f5744"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:28.221475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.221427 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:28.221475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.221474 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/98ae81f5-4281-4ba0-b422-96b5540f5744-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:28.221709 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.221491 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whsk7\" (UniqueName: \"kubernetes.io/projected/98ae81f5-4281-4ba0-b422-96b5540f5744-kube-api-access-whsk7\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:28.875850 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.875822 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" Jul 24 12:25:28.875850 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.875828 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn" event={"ID":"98ae81f5-4281-4ba0-b422-96b5540f5744","Type":"ContainerDied","Data":"3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412"} Jul 24 12:25:28.876292 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:28.875914 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3a73cb22cb1bc87f21ae77c8cc99c6e6ba8dc815d82e67f505f1fb732f531412" Jul 24 12:25:29.016310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.016282 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:29.041563 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.041487 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:29.058135 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.058113 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:29.129041 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129016 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle\") pod \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " Jul 24 12:25:29.129158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129073 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util\") pod \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " Jul 24 12:25:29.129158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129090 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util\") pod \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " Jul 24 12:25:29.129158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129125 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tf2rm\" (UniqueName: \"kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm\") pod \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\" (UID: \"201d23be-aaf9-4abe-a911-b4ec3e37ed0d\") " Jul 24 12:25:29.129332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129157 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n7g4p\" (UniqueName: \"kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p\") pod \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " Jul 24 12:25:29.129332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129185 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle\") pod \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " Jul 24 12:25:29.129332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129236 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle\") pod \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " Jul 24 12:25:29.129332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129279 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util\") pod \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\" (UID: \"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47\") " Jul 24 12:25:29.129493 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129365 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nqbsp\" (UniqueName: \"kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp\") pod \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\" (UID: \"192d104b-c5b9-46ba-8b7f-1e20cfa18300\") " Jul 24 12:25:29.130078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129861 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle" (OuterVolumeSpecName: "bundle") pod "201d23be-aaf9-4abe-a911-b4ec3e37ed0d" (UID: "201d23be-aaf9-4abe-a911-b4ec3e37ed0d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.130078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.129888 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle" (OuterVolumeSpecName: "bundle") pod "192d104b-c5b9-46ba-8b7f-1e20cfa18300" (UID: "192d104b-c5b9-46ba-8b7f-1e20cfa18300"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.130268 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.130228 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle" (OuterVolumeSpecName: "bundle") pod "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" (UID: "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.132168 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.132124 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p" (OuterVolumeSpecName: "kube-api-access-n7g4p") pod "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" (UID: "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47"). InnerVolumeSpecName "kube-api-access-n7g4p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:25:29.132320 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.132174 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp" (OuterVolumeSpecName: "kube-api-access-nqbsp") pod "192d104b-c5b9-46ba-8b7f-1e20cfa18300" (UID: "192d104b-c5b9-46ba-8b7f-1e20cfa18300"). InnerVolumeSpecName "kube-api-access-nqbsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:25:29.132384 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.132324 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm" (OuterVolumeSpecName: "kube-api-access-tf2rm") pod "201d23be-aaf9-4abe-a911-b4ec3e37ed0d" (UID: "201d23be-aaf9-4abe-a911-b4ec3e37ed0d"). InnerVolumeSpecName "kube-api-access-tf2rm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:25:29.135775 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.135747 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util" (OuterVolumeSpecName: "util") pod "192d104b-c5b9-46ba-8b7f-1e20cfa18300" (UID: "192d104b-c5b9-46ba-8b7f-1e20cfa18300"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.135882 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.135859 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util" (OuterVolumeSpecName: "util") pod "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" (UID: "7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.136399 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.136370 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util" (OuterVolumeSpecName: "util") pod "201d23be-aaf9-4abe-a911-b4ec3e37ed0d" (UID: "201d23be-aaf9-4abe-a911-b4ec3e37ed0d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:25:29.230446 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230408 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230446 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230441 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230446 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230453 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nqbsp\" (UniqueName: \"kubernetes.io/projected/192d104b-c5b9-46ba-8b7f-1e20cfa18300-kube-api-access-nqbsp\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230464 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230472 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230479 2578 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-util\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230488 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tf2rm\" (UniqueName: \"kubernetes.io/projected/201d23be-aaf9-4abe-a911-b4ec3e37ed0d-kube-api-access-tf2rm\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230496 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n7g4p\" (UniqueName: \"kubernetes.io/projected/7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47-kube-api-access-n7g4p\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.230695 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.230503 2578 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/192d104b-c5b9-46ba-8b7f-1e20cfa18300-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:25:29.882117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.882090 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" Jul 24 12:25:29.882117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.882099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz" event={"ID":"192d104b-c5b9-46ba-8b7f-1e20cfa18300","Type":"ContainerDied","Data":"2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2"} Jul 24 12:25:29.882564 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.882151 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2ab9ebd2ce75ab1dfdb30633f6c859b953f5b3863011957da2c1095eb4f0a5f2" Jul 24 12:25:29.884055 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.884036 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" Jul 24 12:25:29.884259 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.884034 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc" event={"ID":"201d23be-aaf9-4abe-a911-b4ec3e37ed0d","Type":"ContainerDied","Data":"3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24"} Jul 24 12:25:29.884259 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.884158 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bfbe2d4af50df4117404b19309291f67f269e9089ccde8b647aa6053b965e24" Jul 24 12:25:29.886049 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.886017 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" event={"ID":"7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47","Type":"ContainerDied","Data":"c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809"} Jul 24 12:25:29.886049 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.886049 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="c76da4a2f64eca4811992ef6c3667d5b523bbf02a281b41fc66c420e01f14809" Jul 24 12:25:29.886212 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:29.886067 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m" Jul 24 12:25:34.582754 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.582711 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45"] Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.582991 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583001 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583012 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583017 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583026 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583031 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583037 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583041 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583046 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583051 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583057 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583062 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583067 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583071 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583077 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583082 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="extract" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583088 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583093 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583098 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583102 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="pull" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583109 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="util" Jul 24 12:25:34.583108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583113 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="util" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583122 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="extract" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583127 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="extract" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583169 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="98ae81f5-4281-4ba0-b422-96b5540f5744" containerName="extract" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583175 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="201d23be-aaf9-4abe-a911-b4ec3e37ed0d" containerName="extract" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583182 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47" containerName="extract" Jul 24 12:25:34.583730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.583203 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="192d104b-c5b9-46ba-8b7f-1e20cfa18300" containerName="extract" Jul 24 12:25:34.587456 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.587436 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:34.598289 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.598267 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-lpqw8\"" Jul 24 12:25:34.611272 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.611246 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45"] Jul 24 12:25:34.679010 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.678975 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klhbb\" (UniqueName: \"kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb\") pod \"limitador-operator-controller-manager-859d7b78bd-mlw45\" (UID: \"f6c302f4-db54-4bbf-908b-d0a43314961f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:34.780289 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.780254 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klhbb\" (UniqueName: \"kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb\") pod \"limitador-operator-controller-manager-859d7b78bd-mlw45\" (UID: \"f6c302f4-db54-4bbf-908b-d0a43314961f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:34.788512 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.788477 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klhbb\" (UniqueName: \"kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb\") pod \"limitador-operator-controller-manager-859d7b78bd-mlw45\" (UID: \"f6c302f4-db54-4bbf-908b-d0a43314961f\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:34.897088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:34.896989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:35.038902 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:35.038878 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45"] Jul 24 12:25:35.041303 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:35.041276 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6c302f4_db54_4bbf_908b_d0a43314961f.slice/crio-e65ddce7df74e67c97c9590e712764f9d8cb42ad0908f99301bd2d035aa05a47 WatchSource:0}: Error finding container e65ddce7df74e67c97c9590e712764f9d8cb42ad0908f99301bd2d035aa05a47: Status 404 returned error can't find the container with id e65ddce7df74e67c97c9590e712764f9d8cb42ad0908f99301bd2d035aa05a47 Jul 24 12:25:35.918706 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:35.918662 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" event={"ID":"f6c302f4-db54-4bbf-908b-d0a43314961f","Type":"ContainerStarted","Data":"e65ddce7df74e67c97c9590e712764f9d8cb42ad0908f99301bd2d035aa05a47"} Jul 24 12:25:36.880562 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.880481 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c9b9647f6-4vgz8"] Jul 24 12:25:36.883786 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.883766 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.900576 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.900550 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c9b9647f6-4vgz8"] Jul 24 12:25:36.924794 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.924761 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" event={"ID":"f6c302f4-db54-4bbf-908b-d0a43314961f","Type":"ContainerStarted","Data":"0e3a1ed6d8296bdd4c94155f97b99b91bf841865d16e13dad526fbb8cbc567a0"} Jul 24 12:25:36.925133 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.924877 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:36.954025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.953971 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" podStartSLOduration=1.472451471 podStartE2EDuration="2.953956661s" totalImagesPullingTime="1.48150519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:36.951833679 +0000 UTC m=+393.237379386" watchObservedRunningTime="2026-07-24 12:25:36.953956661 +0000 UTC m=+393.239502344" Jul 24 12:25:36.995645 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995617 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995657 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-oauth-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995675 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-trusted-ca-bundle\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995693 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-service-ca\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995797 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-oauth-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995844 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5bz6\" (UniqueName: \"kubernetes.io/projected/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-kube-api-access-s5bz6\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:36.995922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:36.995885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097025 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5bz6\" (UniqueName: \"kubernetes.io/projected/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-kube-api-access-s5bz6\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097070 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097308 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097100 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097308 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-oauth-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097308 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097159 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-trusted-ca-bundle\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097308 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097186 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-service-ca\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097376 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-oauth-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097984 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-service-ca\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097984 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097904 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.097984 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.097943 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-oauth-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.098322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.098105 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-trusted-ca-bundle\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.099829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.099800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-serving-cert\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.099921 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.099808 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-console-oauth-config\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.106770 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.106747 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5bz6\" (UniqueName: \"kubernetes.io/projected/8fa48eeb-c75e-4c4e-9819-cbe862f1588f-kube-api-access-s5bz6\") pod \"console-7c9b9647f6-4vgz8\" (UID: \"8fa48eeb-c75e-4c4e-9819-cbe862f1588f\") " pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.192830 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.192742 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:37.346440 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.343884 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c9b9647f6-4vgz8"] Jul 24 12:25:37.930911 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.930868 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c9b9647f6-4vgz8" event={"ID":"8fa48eeb-c75e-4c4e-9819-cbe862f1588f","Type":"ContainerStarted","Data":"8955b50c1b3793d99a0c3a3adca76cb7eab770507bf86ecdd36c3413679dcfa9"} Jul 24 12:25:37.931336 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.930933 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c9b9647f6-4vgz8" event={"ID":"8fa48eeb-c75e-4c4e-9819-cbe862f1588f","Type":"ContainerStarted","Data":"8740c2e46e3464ecdfda00cc4d30fd060c89727d0e0794091ed5add590a5b293"} Jul 24 12:25:37.956420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:37.956362 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c9b9647f6-4vgz8" podStartSLOduration=1.956345947 podStartE2EDuration="1.956345947s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:37.953978947 +0000 UTC m=+394.239524632" watchObservedRunningTime="2026-07-24 12:25:37.956345947 +0000 UTC m=+394.241891632" Jul 24 12:25:38.686723 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.686687 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d"] Jul 24 12:25:38.689990 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.689974 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:38.693875 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.693854 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-hgnlx\"" Jul 24 12:25:38.701618 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.701593 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d"] Jul 24 12:25:38.813353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.813316 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x26q6\" (UniqueName: \"kubernetes.io/projected/5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036-kube-api-access-x26q6\") pod \"authorino-operator-75cc8d5cb7-h7b9d\" (UID: \"5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:38.914462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.914426 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x26q6\" (UniqueName: \"kubernetes.io/projected/5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036-kube-api-access-x26q6\") pod \"authorino-operator-75cc8d5cb7-h7b9d\" (UID: \"5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:38.926031 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:38.925995 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x26q6\" (UniqueName: \"kubernetes.io/projected/5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036-kube-api-access-x26q6\") pod \"authorino-operator-75cc8d5cb7-h7b9d\" (UID: \"5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:39.000461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:39.000360 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:39.140067 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:39.140037 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d"] Jul 24 12:25:39.142547 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:39.142520 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f08ffa8_d8d2_4fee_bd0c_0fc4fa430036.slice/crio-8a41f696b59381168683a1e9264fe9ab83b24ef9d35a3b3a2eef5f03ea2c1a3d WatchSource:0}: Error finding container 8a41f696b59381168683a1e9264fe9ab83b24ef9d35a3b3a2eef5f03ea2c1a3d: Status 404 returned error can't find the container with id 8a41f696b59381168683a1e9264fe9ab83b24ef9d35a3b3a2eef5f03ea2c1a3d Jul 24 12:25:39.947608 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:39.947565 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" event={"ID":"5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036","Type":"ContainerStarted","Data":"8a41f696b59381168683a1e9264fe9ab83b24ef9d35a3b3a2eef5f03ea2c1a3d"} Jul 24 12:25:41.959831 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:41.959793 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" event={"ID":"5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036","Type":"ContainerStarted","Data":"63c8fc500bb73a8e63bb46562bbaa7cf9b8949230a8c82b15d1329af5d4999d6"} Jul 24 12:25:41.960290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:41.959858 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:41.994257 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:41.994209 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" podStartSLOduration=1.835734577 podStartE2EDuration="3.994177414s" totalImagesPullingTime="2.158442837s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:41.991591364 +0000 UTC m=+398.277137049" watchObservedRunningTime="2026-07-24 12:25:41.994177414 +0000 UTC m=+398.279723099" Jul 24 12:25:42.039142 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.039111 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8"] Jul 24 12:25:42.042534 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.042512 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:42.046696 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.046674 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 24 12:25:42.046815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.046697 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-bz5f4\"" Jul 24 12:25:42.061590 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.059950 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8"] Jul 24 12:25:42.141589 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.141529 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6lrn\" (UniqueName: \"kubernetes.io/projected/9cfcb201-4fd9-4c8f-a5e9-929f948eb45c-kube-api-access-k6lrn\") pod \"dns-operator-controller-manager-864dd78c4c-rpts8\" (UID: \"9cfcb201-4fd9-4c8f-a5e9-929f948eb45c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:42.242902 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.242814 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6lrn\" (UniqueName: \"kubernetes.io/projected/9cfcb201-4fd9-4c8f-a5e9-929f948eb45c-kube-api-access-k6lrn\") pod \"dns-operator-controller-manager-864dd78c4c-rpts8\" (UID: \"9cfcb201-4fd9-4c8f-a5e9-929f948eb45c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:42.254455 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.254425 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6lrn\" (UniqueName: \"kubernetes.io/projected/9cfcb201-4fd9-4c8f-a5e9-929f948eb45c-kube-api-access-k6lrn\") pod \"dns-operator-controller-manager-864dd78c4c-rpts8\" (UID: \"9cfcb201-4fd9-4c8f-a5e9-929f948eb45c\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:42.352433 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.352395 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:42.493916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.493837 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8"] Jul 24 12:25:42.497400 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:42.497369 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cfcb201_4fd9_4c8f_a5e9_929f948eb45c.slice/crio-c2830401b69bb3f18301b324dc098b93e136e2c295f00bd46b874ec54b255390 WatchSource:0}: Error finding container c2830401b69bb3f18301b324dc098b93e136e2c295f00bd46b874ec54b255390: Status 404 returned error can't find the container with id c2830401b69bb3f18301b324dc098b93e136e2c295f00bd46b874ec54b255390 Jul 24 12:25:42.966298 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:42.966261 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" event={"ID":"9cfcb201-4fd9-4c8f-a5e9-929f948eb45c","Type":"ContainerStarted","Data":"c2830401b69bb3f18301b324dc098b93e136e2c295f00bd46b874ec54b255390"} Jul 24 12:25:44.979043 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:44.979011 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" event={"ID":"9cfcb201-4fd9-4c8f-a5e9-929f948eb45c","Type":"ContainerStarted","Data":"53584c9c3eea2aa3d1fc6d756d319e9f2a8d060d22d2b8aecdd77e1a1e33c345"} Jul 24 12:25:44.979421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:44.979129 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:45.005342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:45.005291 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" podStartSLOduration=2.117093656 podStartE2EDuration="4.005275766s" totalImagesPullingTime="1.88818211s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:25:45.00312231 +0000 UTC m=+401.288667994" watchObservedRunningTime="2026-07-24 12:25:45.005275766 +0000 UTC m=+401.290821451" Jul 24 12:25:47.193071 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:47.193033 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:47.193477 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:47.193118 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:47.197753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:47.197725 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:47.933280 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:47.933248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:25:47.998072 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:47.998043 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c9b9647f6-4vgz8" Jul 24 12:25:48.072827 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:48.072792 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:25:52.968869 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:52.968843 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-h7b9d" Jul 24 12:25:55.171647 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.171605 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7"] Jul 24 12:25:55.175103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.175086 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.176964 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.176939 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 24 12:25:55.177146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.177122 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 24 12:25:55.177288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.177211 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-nl2w7\"" Jul 24 12:25:55.182473 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.182407 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7"] Jul 24 12:25:55.257247 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.257216 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.257419 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.257286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z9997\" (UniqueName: \"kubernetes.io/projected/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-kube-api-access-z9997\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.257419 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.257335 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.358565 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.358527 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.358747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.358589 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.358747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.358643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z9997\" (UniqueName: \"kubernetes.io/projected/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-kube-api-access-z9997\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.358820 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:25:55.358751 2578 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 24 12:25:55.358852 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:25:55.358831 2578 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert podName:bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5 nodeName:}" failed. No retries permitted until 2026-07-24 12:25:55.858809398 +0000 UTC m=+412.144355062 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-zqzf7" (UID: "bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5") : secret "plugin-serving-cert" not found Jul 24 12:25:55.359204 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.359171 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.367392 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.367364 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z9997\" (UniqueName: \"kubernetes.io/projected/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-kube-api-access-z9997\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.861251 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.861181 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.863730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.863708 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-zqzf7\" (UID: \"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:55.986350 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:55.986321 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-rpts8" Jul 24 12:25:56.085092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:56.085051 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" Jul 24 12:25:56.250774 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:56.250742 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7"] Jul 24 12:25:56.254062 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:25:56.254035 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbf7ec99f_3b8c_4fa5_a7ba_0d1305dc89b5.slice/crio-a346dd7ac630e4927a232481a3c11ab2cc4ef5b7b73c29d841740d39dc72f61b WatchSource:0}: Error finding container a346dd7ac630e4927a232481a3c11ab2cc4ef5b7b73c29d841740d39dc72f61b: Status 404 returned error can't find the container with id a346dd7ac630e4927a232481a3c11ab2cc4ef5b7b73c29d841740d39dc72f61b Jul 24 12:25:57.038001 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:25:57.037964 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" event={"ID":"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5","Type":"ContainerStarted","Data":"a346dd7ac630e4927a232481a3c11ab2cc4ef5b7b73c29d841740d39dc72f61b"} Jul 24 12:26:05.075586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.075541 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg"] Jul 24 12:26:05.095458 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.095422 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.099170 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.099145 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-vgv4q\"" Jul 24 12:26:05.099550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.099526 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg"] Jul 24 12:26:05.148922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.148886 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.149103 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.148955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvlbt\" (UniqueName: \"kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.250025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.249986 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pvlbt\" (UniqueName: \"kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.250253 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.250135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.250737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.250711 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.261437 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.261408 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvlbt\" (UniqueName: \"kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt\") pod \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.409095 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.409004 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:05.412793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.412761 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg"] Jul 24 12:26:05.424790 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.424748 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg"] Jul 24 12:26:05.450328 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.450297 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:05.455470 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.455445 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.464644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.464234 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45"] Jul 24 12:26:05.464644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.464460 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" containerName="manager" containerID="cri-o://0e3a1ed6d8296bdd4c94155f97b99b91bf841865d16e13dad526fbb8cbc567a0" gracePeriod=2 Jul 24 12:26:05.475941 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.475183 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45"] Jul 24 12:26:05.481303 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.481094 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:05.505091 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.505056 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8"] Jul 24 12:26:05.505536 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.505512 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" containerName="manager" Jul 24 12:26:05.505632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.505540 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" containerName="manager" Jul 24 12:26:05.505632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.505617 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" containerName="manager" Jul 24 12:26:05.508623 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.508598 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:05.518806 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:05.518639 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mlw45\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:26:05.535617 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.535327 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8"] Jul 24 12:26:05.552643 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.552605 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7g6l\" (UniqueName: \"kubernetes.io/projected/92fe4503-a7d9-47f4-8ade-cce49d223dfd-kube-api-access-b7g6l\") pod \"limitador-operator-controller-manager-859d7b78bd-dhxm8\" (UID: \"92fe4503-a7d9-47f4-8ade-cce49d223dfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:05.552805 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.552762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.552867 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.552818 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dh88f\" (UniqueName: \"kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.653683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.653643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7g6l\" (UniqueName: \"kubernetes.io/projected/92fe4503-a7d9-47f4-8ade-cce49d223dfd-kube-api-access-b7g6l\") pod \"limitador-operator-controller-manager-859d7b78bd-dhxm8\" (UID: \"92fe4503-a7d9-47f4-8ade-cce49d223dfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:05.653866 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.653753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.653866 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.653798 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dh88f\" (UniqueName: \"kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.654244 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.654219 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.671174 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.671095 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7g6l\" (UniqueName: \"kubernetes.io/projected/92fe4503-a7d9-47f4-8ade-cce49d223dfd-kube-api-access-b7g6l\") pod \"limitador-operator-controller-manager-859d7b78bd-dhxm8\" (UID: \"92fe4503-a7d9-47f4-8ade-cce49d223dfd\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:05.671823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.671800 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dh88f\" (UniqueName: \"kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f\") pod \"kuadrant-operator-controller-manager-55467d9f7-rbrfl\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.769613 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.769569 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:05.841079 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:05.841045 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:13.091829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:13.091758 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-67b96579f6-52grh" podUID="31e979ae-f0db-43e2-89eb-9afda947dc19" containerName="console" containerID="cri-o://150131a0aed1515b182666d4fc024953618a1cd171091a424240754465753b12" gracePeriod=15 Jul 24 12:26:17.160370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.160160 2578 generic.go:356] "Generic (PLEG): container finished" podID="31e979ae-f0db-43e2-89eb-9afda947dc19" containerID="150131a0aed1515b182666d4fc024953618a1cd171091a424240754465753b12" exitCode=0 Jul 24 12:26:17.160370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.160295 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b96579f6-52grh" event={"ID":"31e979ae-f0db-43e2-89eb-9afda947dc19","Type":"ContainerDied","Data":"150131a0aed1515b182666d4fc024953618a1cd171091a424240754465753b12"} Jul 24 12:26:17.163552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.163498 2578 generic.go:356] "Generic (PLEG): container finished" podID="f6c302f4-db54-4bbf-908b-d0a43314961f" containerID="0e3a1ed6d8296bdd4c94155f97b99b91bf841865d16e13dad526fbb8cbc567a0" exitCode=0 Jul 24 12:26:17.167609 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.167583 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:26:17.188295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.188268 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:26:17.194208 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.194168 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:17.196741 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:17.196695 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mlw45\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:26:17.215251 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.215163 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8"] Jul 24 12:26:17.217767 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:26:17.217742 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod92fe4503_a7d9_47f4_8ade_cce49d223dfd.slice/crio-bc139e29723e683ad2911d1befe98de4ee40948c961802d3bf9323271fdf6283 WatchSource:0}: Error finding container bc139e29723e683ad2911d1befe98de4ee40948c961802d3bf9323271fdf6283: Status 404 returned error can't find the container with id bc139e29723e683ad2911d1befe98de4ee40948c961802d3bf9323271fdf6283 Jul 24 12:26:17.259205 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259141 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-klhbb\" (UniqueName: \"kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb\") pod \"f6c302f4-db54-4bbf-908b-d0a43314961f\" (UID: \"f6c302f4-db54-4bbf-908b-d0a43314961f\") " Jul 24 12:26:17.259340 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259262 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xgqdc\" (UniqueName: \"kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259340 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259299 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259340 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259336 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259376 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259408 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.259499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259446 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config\") pod \"31e979ae-f0db-43e2-89eb-9afda947dc19\" (UID: \"31e979ae-f0db-43e2-89eb-9afda947dc19\") " Jul 24 12:26:17.260017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.259794 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca" (OuterVolumeSpecName: "service-ca") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:26:17.260234 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.260179 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config" (OuterVolumeSpecName: "console-config") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:26:17.260331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.260285 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:26:17.261179 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.261141 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:26:17.261389 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.261369 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb" (OuterVolumeSpecName: "kube-api-access-klhbb") pod "f6c302f4-db54-4bbf-908b-d0a43314961f" (UID: "f6c302f4-db54-4bbf-908b-d0a43314961f"). InnerVolumeSpecName "kube-api-access-klhbb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:26:17.261729 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.261710 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc" (OuterVolumeSpecName: "kube-api-access-xgqdc") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "kube-api-access-xgqdc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:26:17.262166 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.262148 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:26:17.262245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.262166 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "31e979ae-f0db-43e2-89eb-9afda947dc19" (UID: "31e979ae-f0db-43e2-89eb-9afda947dc19"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:26:17.360799 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360764 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xgqdc\" (UniqueName: \"kubernetes.io/projected/31e979ae-f0db-43e2-89eb-9afda947dc19-kube-api-access-xgqdc\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.360799 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360800 2578 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-service-ca\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360814 2578 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-oauth-serving-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360826 2578 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-serving-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360839 2578 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/31e979ae-f0db-43e2-89eb-9afda947dc19-console-oauth-config\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360867 2578 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-trusted-ca-bundle\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360881 2578 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/31e979ae-f0db-43e2-89eb-9afda947dc19-console-config\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:17.361021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:17.360893 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-klhbb\" (UniqueName: \"kubernetes.io/projected/f6c302f4-db54-4bbf-908b-d0a43314961f-kube-api-access-klhbb\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:18.148911 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.148879 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" path="/var/lib/kubelet/pods/f6c302f4-db54-4bbf-908b-d0a43314961f/volumes" Jul 24 12:26:18.169234 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.169179 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" event={"ID":"92fe4503-a7d9-47f4-8ade-cce49d223dfd","Type":"ContainerStarted","Data":"430c7d5916f279177bb1256bf5fd0ba131562051f82fc860a05eded85be6963e"} Jul 24 12:26:18.169564 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.169262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" event={"ID":"92fe4503-a7d9-47f4-8ade-cce49d223dfd","Type":"ContainerStarted","Data":"bc139e29723e683ad2911d1befe98de4ee40948c961802d3bf9323271fdf6283"} Jul 24 12:26:18.169564 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.169317 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:18.170438 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.170418 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" event={"ID":"34c2e214-d262-40ed-9e1d-7346799ab7c4","Type":"ContainerStarted","Data":"43b65c8a2e6b564bb84370223bf9cc683c60008ec39d94ae39bd015155aeaaea"} Jul 24 12:26:18.171762 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.171733 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" event={"ID":"bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5","Type":"ContainerStarted","Data":"7d34a611eb50cc759bda7e4984195c09d2bb18e5b7d57365fa85142d8d58c496"} Jul 24 12:26:18.173142 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.173127 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b96579f6-52grh" Jul 24 12:26:18.173263 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.173140 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b96579f6-52grh" event={"ID":"31e979ae-f0db-43e2-89eb-9afda947dc19","Type":"ContainerDied","Data":"1853f949f1483fe991431d7e7d68a834d6e4728e57de5f4f46b3b1ee2f9eacab"} Jul 24 12:26:18.173263 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.173184 2578 scope.go:117] "RemoveContainer" containerID="150131a0aed1515b182666d4fc024953618a1cd171091a424240754465753b12" Jul 24 12:26:18.174708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.174570 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:26:18.181886 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.181861 2578 scope.go:117] "RemoveContainer" containerID="0e3a1ed6d8296bdd4c94155f97b99b91bf841865d16e13dad526fbb8cbc567a0" Jul 24 12:26:18.187223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.187164 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" podStartSLOduration=13.187150379 podStartE2EDuration="13.187150379s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:26:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:18.186995958 +0000 UTC m=+434.472541644" watchObservedRunningTime="2026-07-24 12:26:18.187150379 +0000 UTC m=+434.472696065" Jul 24 12:26:18.211272 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.211246 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:26:18.221581 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.221554 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67b96579f6-52grh"] Jul 24 12:26:18.266732 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:18.266636 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-mlw45\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="f6c302f4-db54-4bbf-908b-d0a43314961f" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-mlw45" Jul 24 12:26:18.266891 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:18.266817 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-zqzf7" podStartSLOduration=2.41251795 podStartE2EDuration="23.266800197s" totalImagesPullingTime="20.854282247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:18.264971356 +0000 UTC m=+434.550517044" watchObservedRunningTime="2026-07-24 12:26:18.266800197 +0000 UTC m=+434.552345883" Jul 24 12:26:19.550467 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:19.550415 2578 log.go:32] "RunPodSandbox from runtime service failed" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hz7dg_kuadrant-system_83c95bb7-e454-4170-9931-867258440597_0(2f6aca26581eb3a508231b02ef759e163c851c8c891968890959fb9ce8362af5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hz7dg to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" Jul 24 12:26:19.550777 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:19.550500 2578 kuberuntime_sandbox.go:71] "Failed to create sandbox for pod" err="rpc error: code = Unknown desc = failed to create pod network sandbox k8s_kuadrant-operator-controller-manager-55467d9f7-hz7dg_kuadrant-system_83c95bb7-e454-4170-9931-867258440597_0(2f6aca26581eb3a508231b02ef759e163c851c8c891968890959fb9ce8362af5): error adding pod kuadrant-system_kuadrant-operator-controller-manager-55467d9f7-hz7dg to CNI network \"multus-cni-network\": plugin type=\"multus-shim\" name=\"multus-cni-network\" failed (add): CmdAdd (shim): failed to send CNI request: Post \"http://dummy/cni\": EOF: StdinData: {\"auxiliaryCNIChainName\":\"vendor-cni-chain\",\"binDir\":\"/var/lib/cni/bin\",\"clusterNetwork\":\"/host/run/multus/cni/net.d/10-ovn-kubernetes.conf\",\"cniVersion\":\"0.3.1\",\"daemonSocketDir\":\"/run/multus/socket\",\"globalNamespaces\":\"default,openshift-multus,openshift-sriov-network-operator,openshift-cnv\",\"logLevel\":\"verbose\",\"logToStderr\":true,\"name\":\"multus-cni-network\",\"namespaceIsolation\":true,\"type\":\"multus-shim\"}" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:20.153391 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.150547 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31e979ae-f0db-43e2-89eb-9afda947dc19" path="/var/lib/kubelet/pods/31e979ae-f0db-43e2-89eb-9afda947dc19/volumes" Jul 24 12:26:20.188708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.188674 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:20.190595 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:20.190559 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="83c95bb7-e454-4170-9931-867258440597" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:20.197816 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.197792 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:20.199723 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:20.199691 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="83c95bb7-e454-4170-9931-867258440597" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:20.286306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.286266 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvlbt\" (UniqueName: \"kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt\") pod \"83c95bb7-e454-4170-9931-867258440597\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " Jul 24 12:26:20.286487 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.286359 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume\") pod \"83c95bb7-e454-4170-9931-867258440597\" (UID: \"83c95bb7-e454-4170-9931-867258440597\") " Jul 24 12:26:20.286668 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.286636 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "83c95bb7-e454-4170-9931-867258440597" (UID: "83c95bb7-e454-4170-9931-867258440597"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:26:20.288892 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.288863 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt" (OuterVolumeSpecName: "kube-api-access-pvlbt") pod "83c95bb7-e454-4170-9931-867258440597" (UID: "83c95bb7-e454-4170-9931-867258440597"). InnerVolumeSpecName "kube-api-access-pvlbt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:26:20.387849 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.387814 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvlbt\" (UniqueName: \"kubernetes.io/projected/83c95bb7-e454-4170-9931-867258440597-kube-api-access-pvlbt\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:20.387849 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:20.387849 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/83c95bb7-e454-4170-9931-867258440597-extensions-socket-volume\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:21.195602 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:21.195562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" event={"ID":"34c2e214-d262-40ed-9e1d-7346799ab7c4","Type":"ContainerStarted","Data":"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3"} Jul 24 12:26:21.195602 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:21.195601 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:21.196049 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:21.195767 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:21.197424 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:21.197393 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="83c95bb7-e454-4170-9931-867258440597" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:21.221554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:21.221459 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" podStartSLOduration=12.44882886 podStartE2EDuration="16.221442953s" totalImagesPullingTime="3.772614093s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:26:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:21.220690263 +0000 UTC m=+437.506235969" watchObservedRunningTime="2026-07-24 12:26:21.221442953 +0000 UTC m=+437.506988689" Jul 24 12:26:21.222295 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:21.222265 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-55467d9f7-hz7dg\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="83c95bb7-e454-4170-9931-867258440597" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-hz7dg" Jul 24 12:26:22.147736 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:22.147701 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83c95bb7-e454-4170-9931-867258440597" path="/var/lib/kubelet/pods/83c95bb7-e454-4170-9931-867258440597/volumes" Jul 24 12:26:29.185754 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:29.185677 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-dhxm8" Jul 24 12:26:32.202475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:32.202444 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:44.792186 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:44.792148 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:44.792686 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:44.792403 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" podUID="34c2e214-d262-40ed-9e1d-7346799ab7c4" containerName="manager" containerID="cri-o://64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3" gracePeriod=10 Jul 24 12:26:44.846045 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:44.846009 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34c2e214_d262_40ed_9e1d_7346799ab7c4.slice/crio-conmon-64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3.scope\": RecentStats: unable to find data in memory cache]" Jul 24 12:26:45.037597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.037575 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:45.194088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.193993 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume\") pod \"34c2e214-d262-40ed-9e1d-7346799ab7c4\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " Jul 24 12:26:45.194302 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.194094 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dh88f\" (UniqueName: \"kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f\") pod \"34c2e214-d262-40ed-9e1d-7346799ab7c4\" (UID: \"34c2e214-d262-40ed-9e1d-7346799ab7c4\") " Jul 24 12:26:45.194529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.194495 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "34c2e214-d262-40ed-9e1d-7346799ab7c4" (UID: "34c2e214-d262-40ed-9e1d-7346799ab7c4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:26:45.196318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.196294 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f" (OuterVolumeSpecName: "kube-api-access-dh88f") pod "34c2e214-d262-40ed-9e1d-7346799ab7c4" (UID: "34c2e214-d262-40ed-9e1d-7346799ab7c4"). InnerVolumeSpecName "kube-api-access-dh88f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:26:45.295530 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.295496 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dh88f\" (UniqueName: \"kubernetes.io/projected/34c2e214-d262-40ed-9e1d-7346799ab7c4-kube-api-access-dh88f\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:45.295530 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.295521 2578 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/34c2e214-d262-40ed-9e1d-7346799ab7c4-extensions-socket-volume\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:26:45.318711 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.318681 2578 generic.go:356] "Generic (PLEG): container finished" podID="34c2e214-d262-40ed-9e1d-7346799ab7c4" containerID="64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3" exitCode=0 Jul 24 12:26:45.318874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.318724 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" Jul 24 12:26:45.318874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.318737 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" event={"ID":"34c2e214-d262-40ed-9e1d-7346799ab7c4","Type":"ContainerDied","Data":"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3"} Jul 24 12:26:45.318874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.318780 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl" event={"ID":"34c2e214-d262-40ed-9e1d-7346799ab7c4","Type":"ContainerDied","Data":"43b65c8a2e6b564bb84370223bf9cc683c60008ec39d94ae39bd015155aeaaea"} Jul 24 12:26:45.318874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.318801 2578 scope.go:117] "RemoveContainer" containerID="64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3" Jul 24 12:26:45.327778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.327760 2578 scope.go:117] "RemoveContainer" containerID="64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3" Jul 24 12:26:45.328023 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:26:45.328002 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3\": container with ID starting with 64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3 not found: ID does not exist" containerID="64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3" Jul 24 12:26:45.328064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.328032 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3"} err="failed to get container status \"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3\": rpc error: code = NotFound desc = could not find container \"64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3\": container with ID starting with 64d57dd3eaf3a632059f1e3640ecb9cba459c03b92cbf12848601b38d2bed9b3 not found: ID does not exist" Jul 24 12:26:45.341398 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.341367 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:45.346022 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:45.345999 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-55467d9f7-rbrfl"] Jul 24 12:26:46.150910 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:46.150868 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="34c2e214-d262-40ed-9e1d-7346799ab7c4" path="/var/lib/kubelet/pods/34c2e214-d262-40ed-9e1d-7346799ab7c4/volumes" Jul 24 12:26:48.655559 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655517 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm"] Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655815 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31e979ae-f0db-43e2-89eb-9afda947dc19" containerName="console" Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655826 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="31e979ae-f0db-43e2-89eb-9afda947dc19" containerName="console" Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655843 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="34c2e214-d262-40ed-9e1d-7346799ab7c4" containerName="manager" Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655849 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="34c2e214-d262-40ed-9e1d-7346799ab7c4" containerName="manager" Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655901 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="34c2e214-d262-40ed-9e1d-7346799ab7c4" containerName="manager" Jul 24 12:26:48.656011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.655910 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="31e979ae-f0db-43e2-89eb-9afda947dc19" containerName="console" Jul 24 12:26:48.802854 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.802813 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm"] Jul 24 12:26:48.803011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.802934 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.805241 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.805223 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-48tdf\"" Jul 24 12:26:48.931630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931536 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931595 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ddf789-0df8-4c83-b607-33d3954167e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931619 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ddf789-0df8-4c83-b607-33d3954167e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931666 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931794 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjwlg\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-kube-api-access-jjwlg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:48.931929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:48.931876 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.032779 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.032779 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032779 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032813 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032831 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jjwlg\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-kube-api-access-jjwlg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032855 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032895 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ddf789-0df8-4c83-b607-33d3954167e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032968 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033025 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.032994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ddf789-0df8-4c83-b607-33d3954167e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.033163 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.033363 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033507 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.033462 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.033512 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.033662 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.033645 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/59ddf789-0df8-4c83-b607-33d3954167e4-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.035402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.035381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/59ddf789-0df8-4c83-b607-33d3954167e4-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.035530 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.035513 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/59ddf789-0df8-4c83-b607-33d3954167e4-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.040707 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.040684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.041396 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.041379 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjwlg\" (UniqueName: \"kubernetes.io/projected/59ddf789-0df8-4c83-b607-33d3954167e4-kube-api-access-jjwlg\") pod \"maas-default-gateway-openshift-default-78bb78ff78-hpjpm\" (UID: \"59ddf789-0df8-4c83-b607-33d3954167e4\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.112928 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.112896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:49.348967 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.348940 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm"] Jul 24 12:26:49.351321 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:26:49.351288 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59ddf789_0df8_4c83_b607_33d3954167e4.slice/crio-dfab3eb0b3ea9767577f5c32d63c83276ac30a5b27192efc87bd90df5b467d4e WatchSource:0}: Error finding container dfab3eb0b3ea9767577f5c32d63c83276ac30a5b27192efc87bd90df5b467d4e: Status 404 returned error can't find the container with id dfab3eb0b3ea9767577f5c32d63c83276ac30a5b27192efc87bd90df5b467d4e Jul 24 12:26:49.353317 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.353281 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:26:49.353421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.353358 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:26:49.353421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:49.353391 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:26:50.348619 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:50.348582 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" event={"ID":"59ddf789-0df8-4c83-b607-33d3954167e4","Type":"ContainerStarted","Data":"49c715d40fcdd7b09c706a6d0e2c35c308282c7a30f7e49a3bf989d274db6f9f"} Jul 24 12:26:50.349012 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:50.348635 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" event={"ID":"59ddf789-0df8-4c83-b607-33d3954167e4","Type":"ContainerStarted","Data":"dfab3eb0b3ea9767577f5c32d63c83276ac30a5b27192efc87bd90df5b467d4e"} Jul 24 12:26:50.368235 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:50.368176 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" podStartSLOduration=2.368160492 podStartE2EDuration="2.368160492s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:26:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:50.366075842 +0000 UTC m=+466.651621528" watchObservedRunningTime="2026-07-24 12:26:50.368160492 +0000 UTC m=+466.653706251" Jul 24 12:26:51.113585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:51.113545 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:51.118497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:51.118470 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:51.353868 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:51.353836 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:51.354748 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:51.354733 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:26:52.276526 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.276490 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:26:52.279222 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.279179 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:52.280849 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.280828 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-j658h\"" Jul 24 12:26:52.288102 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.288078 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:26:52.363333 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.363301 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48vsn\" (UniqueName: \"kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn\") pod \"authorino-58cd955675-kmn9x\" (UID: \"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb\") " pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:52.464493 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.464460 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48vsn\" (UniqueName: \"kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn\") pod \"authorino-58cd955675-kmn9x\" (UID: \"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb\") " pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:52.473491 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.473461 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48vsn\" (UniqueName: \"kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn\") pod \"authorino-58cd955675-kmn9x\" (UID: \"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb\") " pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:52.490400 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.490370 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:26:52.492844 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.492825 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:26:52.501167 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.501138 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:26:52.565270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.565165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p684r\" (UniqueName: \"kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r\") pod \"authorino-75d556bd77-rnf5p\" (UID: \"e859fd9b-7a18-4497-8887-38f89d87a912\") " pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:26:52.589733 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.589701 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:52.666538 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.666502 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p684r\" (UniqueName: \"kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r\") pod \"authorino-75d556bd77-rnf5p\" (UID: \"e859fd9b-7a18-4497-8887-38f89d87a912\") " pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:26:52.675226 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.675167 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p684r\" (UniqueName: \"kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r\") pod \"authorino-75d556bd77-rnf5p\" (UID: \"e859fd9b-7a18-4497-8887-38f89d87a912\") " pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:26:52.727700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.727676 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:26:52.729768 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:26:52.729733 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e21b27f_73d9_4a29_ae55_2b69c5d9d7cb.slice/crio-48a61c7ea97b818fcbda8aa0b00c4c9de989c551715234c1c252fc127ed38d22 WatchSource:0}: Error finding container 48a61c7ea97b818fcbda8aa0b00c4c9de989c551715234c1c252fc127ed38d22: Status 404 returned error can't find the container with id 48a61c7ea97b818fcbda8aa0b00c4c9de989c551715234c1c252fc127ed38d22 Jul 24 12:26:52.802713 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.802674 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:26:52.936226 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:52.936179 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:26:52.938399 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:26:52.938374 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode859fd9b_7a18_4497_8887_38f89d87a912.slice/crio-1dfdfb482c87465b9ce9ccbdd07117eb38c58dbf9d91bb1554b0546b7d4d35d4 WatchSource:0}: Error finding container 1dfdfb482c87465b9ce9ccbdd07117eb38c58dbf9d91bb1554b0546b7d4d35d4: Status 404 returned error can't find the container with id 1dfdfb482c87465b9ce9ccbdd07117eb38c58dbf9d91bb1554b0546b7d4d35d4 Jul 24 12:26:53.366213 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:53.366139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kmn9x" event={"ID":"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb","Type":"ContainerStarted","Data":"48a61c7ea97b818fcbda8aa0b00c4c9de989c551715234c1c252fc127ed38d22"} Jul 24 12:26:53.367758 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:53.367730 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rnf5p" event={"ID":"e859fd9b-7a18-4497-8887-38f89d87a912","Type":"ContainerStarted","Data":"1dfdfb482c87465b9ce9ccbdd07117eb38c58dbf9d91bb1554b0546b7d4d35d4"} Jul 24 12:26:57.392994 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:57.392951 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rnf5p" event={"ID":"e859fd9b-7a18-4497-8887-38f89d87a912","Type":"ContainerStarted","Data":"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e"} Jul 24 12:26:57.394384 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:57.394357 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kmn9x" event={"ID":"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb","Type":"ContainerStarted","Data":"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72"} Jul 24 12:26:57.408659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:57.408603 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-rnf5p" podStartSLOduration=1.90224868 podStartE2EDuration="5.408587269s" totalImagesPullingTime="3.506338589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:57.407732269 +0000 UTC m=+473.693277954" watchObservedRunningTime="2026-07-24 12:26:57.408587269 +0000 UTC m=+473.694132932" Jul 24 12:26:57.454792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:57.454740 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-58cd955675-kmn9x" podStartSLOduration=1.7510507849999999 podStartE2EDuration="5.454722347s" totalImagesPullingTime="3.703671562s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:26:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:26:57.43010631 +0000 UTC m=+473.715651995" watchObservedRunningTime="2026-07-24 12:26:57.454722347 +0000 UTC m=+473.740268031" Jul 24 12:26:57.456318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:57.456300 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:26:59.403212 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:59.403155 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-58cd955675-kmn9x" podUID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" containerName="authorino" containerID="cri-o://ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72" gracePeriod=30 Jul 24 12:26:59.654312 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:59.654253 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:26:59.735081 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:59.735046 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48vsn\" (UniqueName: \"kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn\") pod \"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb\" (UID: \"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb\") " Jul 24 12:26:59.737281 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:59.737253 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn" (OuterVolumeSpecName: "kube-api-access-48vsn") pod "4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" (UID: "4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb"). InnerVolumeSpecName "kube-api-access-48vsn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:26:59.835840 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:26:59.835812 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48vsn\" (UniqueName: \"kubernetes.io/projected/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb-kube-api-access-48vsn\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:27:00.409146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.409108 2578 generic.go:356] "Generic (PLEG): container finished" podID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" containerID="ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72" exitCode=0 Jul 24 12:27:00.409554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.409165 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-58cd955675-kmn9x" Jul 24 12:27:00.409554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.409187 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kmn9x" event={"ID":"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb","Type":"ContainerDied","Data":"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72"} Jul 24 12:27:00.409554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.409245 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-58cd955675-kmn9x" event={"ID":"4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb","Type":"ContainerDied","Data":"48a61c7ea97b818fcbda8aa0b00c4c9de989c551715234c1c252fc127ed38d22"} Jul 24 12:27:00.409554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.409269 2578 scope.go:117] "RemoveContainer" containerID="ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72" Jul 24 12:27:00.417782 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.417766 2578 scope.go:117] "RemoveContainer" containerID="ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72" Jul 24 12:27:00.418042 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:27:00.418022 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72\": container with ID starting with ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72 not found: ID does not exist" containerID="ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72" Jul 24 12:27:00.418096 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.418053 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72"} err="failed to get container status \"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72\": rpc error: code = NotFound desc = could not find container \"ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72\": container with ID starting with ba35fdf417312f1a222ee25d94f71606a2d5f43b249723c4369b010d04bace72 not found: ID does not exist" Jul 24 12:27:00.424147 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.424122 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:27:00.427950 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:00.427929 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-58cd955675-kmn9x"] Jul 24 12:27:02.148161 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:02.148128 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" path="/var/lib/kubelet/pods/4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb/volumes" Jul 24 12:27:25.864169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.864126 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgvqk"] Jul 24 12:27:25.864691 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.864628 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" containerName="authorino" Jul 24 12:27:25.864691 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.864649 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" containerName="authorino" Jul 24 12:27:25.864795 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.864745 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e21b27f-73d9-4a29-ae55-2b69c5d9d7cb" containerName="authorino" Jul 24 12:27:25.867462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.867441 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:25.887444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.887415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgvqk"] Jul 24 12:27:25.970203 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:25.970162 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v7z9n\" (UniqueName: \"kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n\") pod \"authorino-5d4b67c446-tgvqk\" (UID: \"fdab1f76-e871-48fc-b254-104d1e7da8d0\") " pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:26.024687 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.024648 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgvqk"] Jul 24 12:27:26.025465 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:27:26.025429 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-v7z9n], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-5d4b67c446-tgvqk" podUID="fdab1f76-e871-48fc-b254-104d1e7da8d0" Jul 24 12:27:26.048390 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.048356 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-76877994-m987j"] Jul 24 12:27:26.051644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.051628 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.054445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.054419 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 24 12:27:26.061568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.061546 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-76877994-m987j"] Jul 24 12:27:26.071471 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.071445 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v7z9n\" (UniqueName: \"kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n\") pod \"authorino-5d4b67c446-tgvqk\" (UID: \"fdab1f76-e871-48fc-b254-104d1e7da8d0\") " pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:26.082270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.082243 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v7z9n\" (UniqueName: \"kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n\") pod \"authorino-5d4b67c446-tgvqk\" (UID: \"fdab1f76-e871-48fc-b254-104d1e7da8d0\") " pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:26.114968 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.114892 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-m987j"] Jul 24 12:27:26.115169 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:27:26.115150 2578 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-xnnb4 tls-cert], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/authorino-76877994-m987j" podUID="d0a962ad-9979-4f02-8166-cb6de5454e88" Jul 24 12:27:26.141791 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.141766 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:27:26.145267 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.145242 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.152311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.152285 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:27:26.172371 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.172332 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnnb4\" (UniqueName: \"kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.172511 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.172403 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.273949 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.273904 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.274126 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.274031 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rbb4j\" (UniqueName: \"kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.274126 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.274064 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.274323 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.274294 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnnb4\" (UniqueName: \"kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.276567 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.276545 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.282105 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.282085 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnnb4\" (UniqueName: \"kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4\") pod \"authorino-76877994-m987j\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.375176 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.375085 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rbb4j\" (UniqueName: \"kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.375176 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.375125 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.377649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.377619 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.382780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.382762 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rbb4j\" (UniqueName: \"kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j\") pod \"authorino-6dd767fcf6-szcjr\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.455521 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.455490 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:27:26.539427 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.539404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:26.539517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.539404 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.549002 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.548922 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:26.555414 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.555382 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:26.591119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.591086 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:27:26.593985 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:27:26.593956 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9305d35d_b35b_4f4e_a6ce_1dff21a355b1.slice/crio-111872482adf703f967e7e66275153ca87eefd4adec96fa0d812270d3da3e845 WatchSource:0}: Error finding container 111872482adf703f967e7e66275153ca87eefd4adec96fa0d812270d3da3e845: Status 404 returned error can't find the container with id 111872482adf703f967e7e66275153ca87eefd4adec96fa0d812270d3da3e845 Jul 24 12:27:26.678075 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.677997 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert\") pod \"d0a962ad-9979-4f02-8166-cb6de5454e88\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " Jul 24 12:27:26.678075 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.678047 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnnb4\" (UniqueName: \"kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4\") pod \"d0a962ad-9979-4f02-8166-cb6de5454e88\" (UID: \"d0a962ad-9979-4f02-8166-cb6de5454e88\") " Jul 24 12:27:26.678075 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.678065 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v7z9n\" (UniqueName: \"kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n\") pod \"fdab1f76-e871-48fc-b254-104d1e7da8d0\" (UID: \"fdab1f76-e871-48fc-b254-104d1e7da8d0\") " Jul 24 12:27:26.680449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.680416 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n" (OuterVolumeSpecName: "kube-api-access-v7z9n") pod "fdab1f76-e871-48fc-b254-104d1e7da8d0" (UID: "fdab1f76-e871-48fc-b254-104d1e7da8d0"). InnerVolumeSpecName "kube-api-access-v7z9n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:27:26.680568 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.680461 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "d0a962ad-9979-4f02-8166-cb6de5454e88" (UID: "d0a962ad-9979-4f02-8166-cb6de5454e88"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:27:26.680620 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.680597 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4" (OuterVolumeSpecName: "kube-api-access-xnnb4") pod "d0a962ad-9979-4f02-8166-cb6de5454e88" (UID: "d0a962ad-9979-4f02-8166-cb6de5454e88"). InnerVolumeSpecName "kube-api-access-xnnb4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:27:26.779118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.779086 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d0a962ad-9979-4f02-8166-cb6de5454e88-tls-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:27:26.779118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.779112 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnnb4\" (UniqueName: \"kubernetes.io/projected/d0a962ad-9979-4f02-8166-cb6de5454e88-kube-api-access-xnnb4\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:27:26.779118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:26.779121 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v7z9n\" (UniqueName: \"kubernetes.io/projected/fdab1f76-e871-48fc-b254-104d1e7da8d0-kube-api-access-v7z9n\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:27:27.545786 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.545748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" event={"ID":"9305d35d-b35b-4f4e-a6ce-1dff21a355b1","Type":"ContainerStarted","Data":"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95"} Jul 24 12:27:27.546164 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.545795 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-76877994-m987j" Jul 24 12:27:27.546164 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.545757 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-5d4b67c446-tgvqk" Jul 24 12:27:27.546164 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.545799 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" event={"ID":"9305d35d-b35b-4f4e-a6ce-1dff21a355b1","Type":"ContainerStarted","Data":"111872482adf703f967e7e66275153ca87eefd4adec96fa0d812270d3da3e845"} Jul 24 12:27:27.563339 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.563296 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" podStartSLOduration=1.163856465 podStartE2EDuration="1.563282222s" totalImagesPullingTime="399.425757ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:27:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:27:27.560992344 +0000 UTC m=+503.846538028" watchObservedRunningTime="2026-07-24 12:27:27.563282222 +0000 UTC m=+503.848827907" Jul 24 12:27:27.593911 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.593865 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgvqk"] Jul 24 12:27:27.599856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.599823 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:27:27.600021 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.600004 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-rnf5p" podUID="e859fd9b-7a18-4497-8887-38f89d87a912" containerName="authorino" containerID="cri-o://2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e" gracePeriod=30 Jul 24 12:27:27.602175 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.602151 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-5d4b67c446-tgvqk"] Jul 24 12:27:27.626372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.626335 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-76877994-m987j"] Jul 24 12:27:27.629426 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.629400 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-76877994-m987j"] Jul 24 12:27:27.872295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.872272 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:27:27.989846 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.989805 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p684r\" (UniqueName: \"kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r\") pod \"e859fd9b-7a18-4497-8887-38f89d87a912\" (UID: \"e859fd9b-7a18-4497-8887-38f89d87a912\") " Jul 24 12:27:27.992045 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:27.992012 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r" (OuterVolumeSpecName: "kube-api-access-p684r") pod "e859fd9b-7a18-4497-8887-38f89d87a912" (UID: "e859fd9b-7a18-4497-8887-38f89d87a912"). InnerVolumeSpecName "kube-api-access-p684r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:27:28.090398 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.090357 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p684r\" (UniqueName: \"kubernetes.io/projected/e859fd9b-7a18-4497-8887-38f89d87a912-kube-api-access-p684r\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:27:28.148135 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.148048 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d0a962ad-9979-4f02-8166-cb6de5454e88" path="/var/lib/kubelet/pods/d0a962ad-9979-4f02-8166-cb6de5454e88/volumes" Jul 24 12:27:28.148322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.148305 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fdab1f76-e871-48fc-b254-104d1e7da8d0" path="/var/lib/kubelet/pods/fdab1f76-e871-48fc-b254-104d1e7da8d0/volumes" Jul 24 12:27:28.552597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.552556 2578 generic.go:356] "Generic (PLEG): container finished" podID="e859fd9b-7a18-4497-8887-38f89d87a912" containerID="2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e" exitCode=0 Jul 24 12:27:28.553013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.552622 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-rnf5p" Jul 24 12:27:28.553013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.552647 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rnf5p" event={"ID":"e859fd9b-7a18-4497-8887-38f89d87a912","Type":"ContainerDied","Data":"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e"} Jul 24 12:27:28.553013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.552701 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-rnf5p" event={"ID":"e859fd9b-7a18-4497-8887-38f89d87a912","Type":"ContainerDied","Data":"1dfdfb482c87465b9ce9ccbdd07117eb38c58dbf9d91bb1554b0546b7d4d35d4"} Jul 24 12:27:28.553013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.552721 2578 scope.go:117] "RemoveContainer" containerID="2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e" Jul 24 12:27:28.566373 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.566344 2578 scope.go:117] "RemoveContainer" containerID="2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e" Jul 24 12:27:28.566808 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:27:28.566784 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e\": container with ID starting with 2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e not found: ID does not exist" containerID="2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e" Jul 24 12:27:28.566870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.566815 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e"} err="failed to get container status \"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e\": rpc error: code = NotFound desc = could not find container \"2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e\": container with ID starting with 2bce9655c68f17750a6a641b95e23a9cae21b9f25997093758c8adb07d754b3e not found: ID does not exist" Jul 24 12:27:28.573975 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.573944 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:27:28.575891 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:28.575868 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-rnf5p"] Jul 24 12:27:30.147701 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:27:30.147664 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e859fd9b-7a18-4497-8887-38f89d87a912" path="/var/lib/kubelet/pods/e859fd9b-7a18-4497-8887-38f89d87a912/volumes" Jul 24 12:28:16.790142 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.790107 2578 generic.go:356] "Generic (PLEG): container finished" podID="59ddf789-0df8-4c83-b607-33d3954167e4" containerID="49c715d40fcdd7b09c706a6d0e2c35c308282c7a30f7e49a3bf989d274db6f9f" exitCode=0 Jul 24 12:28:16.790142 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.790145 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" event={"ID":"59ddf789-0df8-4c83-b607-33d3954167e4","Type":"ContainerDied","Data":"49c715d40fcdd7b09c706a6d0e2c35c308282c7a30f7e49a3bf989d274db6f9f"} Jul 24 12:28:16.790620 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.790560 2578 scope.go:117] "RemoveContainer" containerID="49c715d40fcdd7b09c706a6d0e2c35c308282c7a30f7e49a3bf989d274db6f9f" Jul 24 12:28:16.791647 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.791616 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:28:16.791721 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.791704 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:28:16.791769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:16.791754 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:28:17.797015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:17.796983 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" event={"ID":"59ddf789-0df8-4c83-b607-33d3954167e4","Type":"ContainerStarted","Data":"d0eaf84bcca82415bbee6ba06a5d6ecd52768239d00eec09778aec98b7722f79"} Jul 24 12:28:18.113501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:18.113398 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:28:18.114962 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:18.114929 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hpjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 12:28:18.115255 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:18.115220 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" podUID="59ddf789-0df8-4c83-b607-33d3954167e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 12:28:19.114742 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:19.114702 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hpjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 12:28:19.115780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:19.115318 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" podUID="59ddf789-0df8-4c83-b607-33d3954167e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 12:28:20.114007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:20.113538 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hpjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 12:28:20.114007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:20.113606 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" podUID="59ddf789-0df8-4c83-b607-33d3954167e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 12:28:21.114326 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:21.114283 2578 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-hpjpm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body= Jul 24 12:28:21.114939 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:21.114899 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" podUID="59ddf789-0df8-4c83-b607-33d3954167e4" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" Jul 24 12:28:21.353844 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:21.353795 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:28:22.117171 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:22.117137 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:28:22.118379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:22.118359 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-hpjpm" Jul 24 12:28:26.533004 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.532972 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h"] Jul 24 12:28:26.533381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.533304 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e859fd9b-7a18-4497-8887-38f89d87a912" containerName="authorino" Jul 24 12:28:26.533381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.533315 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e859fd9b-7a18-4497-8887-38f89d87a912" containerName="authorino" Jul 24 12:28:26.533381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.533361 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e859fd9b-7a18-4497-8887-38f89d87a912" containerName="authorino" Jul 24 12:28:26.536174 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.536157 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.537916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.537894 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 24 12:28:26.538024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.537914 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 24 12:28:26.538315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.538294 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-6spvv\"" Jul 24 12:28:26.538315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.538306 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 24 12:28:26.544474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.544451 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h"] Jul 24 12:28:26.665154 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665121 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84afbe14-8890-4a80-ae3f-40340f25b455-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.665331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665176 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.665331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.665331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665260 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.665331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665287 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.665481 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.665353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npfjh\" (UniqueName: \"kubernetes.io/projected/84afbe14-8890-4a80-ae3f-40340f25b455-kube-api-access-npfjh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766515 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766698 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766528 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-npfjh\" (UniqueName: \"kubernetes.io/projected/84afbe14-8890-4a80-ae3f-40340f25b455-kube-api-access-npfjh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766698 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766583 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84afbe14-8890-4a80-ae3f-40340f25b455-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766698 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766698 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766651 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766698 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766668 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.766978 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766946 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.767055 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.766965 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.767110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.767083 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.769113 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.769084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/84afbe14-8890-4a80-ae3f-40340f25b455-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.769277 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.769262 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/84afbe14-8890-4a80-ae3f-40340f25b455-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.774473 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.774454 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-npfjh\" (UniqueName: \"kubernetes.io/projected/84afbe14-8890-4a80-ae3f-40340f25b455-kube-api-access-npfjh\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h\" (UID: \"84afbe14-8890-4a80-ae3f-40340f25b455\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.847306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.847187 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:26.983560 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:26.983533 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h"] Jul 24 12:28:26.985571 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:28:26.985545 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84afbe14_8890_4a80_ae3f_40340f25b455.slice/crio-dbfc55ae6ea67c32e71a3d9f1c0540ea9399ee869532276942292d1cc9b83315 WatchSource:0}: Error finding container dbfc55ae6ea67c32e71a3d9f1c0540ea9399ee869532276942292d1cc9b83315: Status 404 returned error can't find the container with id dbfc55ae6ea67c32e71a3d9f1c0540ea9399ee869532276942292d1cc9b83315 Jul 24 12:28:27.858398 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:27.858349 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" event={"ID":"84afbe14-8890-4a80-ae3f-40340f25b455","Type":"ContainerStarted","Data":"dbfc55ae6ea67c32e71a3d9f1c0540ea9399ee869532276942292d1cc9b83315"} Jul 24 12:28:28.865434 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:28.865393 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" event={"ID":"84afbe14-8890-4a80-ae3f-40340f25b455","Type":"ContainerStarted","Data":"87ce1fd72880281dec8a3b8cb1d82e219e975d0b868259b73937b16ed3c02017"} Jul 24 12:28:28.882897 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:28.882851 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" podStartSLOduration=1.8545343330000001 podStartE2EDuration="2.882834306s" totalImagesPullingTime="1.028299973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:28:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:28:28.881457583 +0000 UTC m=+565.167003267" watchObservedRunningTime="2026-07-24 12:28:28.882834306 +0000 UTC m=+565.168379991" Jul 24 12:28:36.848024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:36.847988 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:36.848468 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:36.848047 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:36.861045 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:36.861014 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:36.915502 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:36.915473 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h" Jul 24 12:28:48.131419 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.131383 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg"] Jul 24 12:28:48.135899 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.135878 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.137943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.137920 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 24 12:28:48.149028 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.149002 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg"] Jul 24 12:28:48.254461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254430 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kpvj\" (UniqueName: \"kubernetes.io/projected/1e600707-b82c-40cf-96eb-0096fd0a2e4b-kube-api-access-5kpvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.254461 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254472 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.254686 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254551 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.254686 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254630 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.254686 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254678 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.254787 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.254717 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kpvj\" (UniqueName: \"kubernetes.io/projected/1e600707-b82c-40cf-96eb-0096fd0a2e4b-kube-api-access-5kpvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355800 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355640 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355800 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355671 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355800 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355715 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355932 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355839 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.355932 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.355896 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.356110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.356091 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.356182 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.356160 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.356276 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.356217 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.358282 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.358261 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/1e600707-b82c-40cf-96eb-0096fd0a2e4b-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.358527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.358511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/1e600707-b82c-40cf-96eb-0096fd0a2e4b-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.363685 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.363660 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kpvj\" (UniqueName: \"kubernetes.io/projected/1e600707-b82c-40cf-96eb-0096fd0a2e4b-kube-api-access-5kpvj\") pod \"e2e-trlp-test-simulated-kserve-598466548d-pgbmg\" (UID: \"1e600707-b82c-40cf-96eb-0096fd0a2e4b\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.446896 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.446801 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:48.585155 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.585132 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg"] Jul 24 12:28:48.587128 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:28:48.587083 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e600707_b82c_40cf_96eb_0096fd0a2e4b.slice/crio-ab3cdac1eb05cd3cace6f8e5e87719fd60028421279e225575832b6e69235a7f WatchSource:0}: Error finding container ab3cdac1eb05cd3cace6f8e5e87719fd60028421279e225575832b6e69235a7f: Status 404 returned error can't find the container with id ab3cdac1eb05cd3cace6f8e5e87719fd60028421279e225575832b6e69235a7f Jul 24 12:28:48.970465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.970430 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" event={"ID":"1e600707-b82c-40cf-96eb-0096fd0a2e4b","Type":"ContainerStarted","Data":"b4764ee549ad906ba84638a9dde689c5cc22310542493c062d351b883b0b8ad6"} Jul 24 12:28:48.970604 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.970481 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" event={"ID":"1e600707-b82c-40cf-96eb-0096fd0a2e4b","Type":"ContainerStarted","Data":"ab3cdac1eb05cd3cace6f8e5e87719fd60028421279e225575832b6e69235a7f"} Jul 24 12:28:48.989325 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:48.989268 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" podStartSLOduration=0.816665728 podStartE2EDuration="989.253418ms" totalImagesPullingTime="172.58769ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:28:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:28:48.986710727 +0000 UTC m=+585.272256412" watchObservedRunningTime="2026-07-24 12:28:48.989253418 +0000 UTC m=+585.274799103" Jul 24 12:28:58.447740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:58.447704 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:58.448119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:58.447766 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:58.459668 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:58.459645 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:28:59.035654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:28:59.035627 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-pgbmg" Jul 24 12:30:00.148780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.148747 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd"] Jul 24 12:30:00.151966 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.151948 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" Jul 24 12:30:00.154279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.154257 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zw8hg\"" Jul 24 12:30:00.157453 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.157427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd"] Jul 24 12:30:00.163769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.163748 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" Jul 24 12:30:00.309838 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.309804 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd"] Jul 24 12:30:00.313369 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:30:00.313331 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eacb9b8_d9b3_4e3e_8e8e_4c5aafd2840b.slice/crio-baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4 WatchSource:0}: Error finding container baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4: Status 404 returned error can't find the container with id baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4 Jul 24 12:30:00.315160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.315144 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 12:30:00.338352 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:00.338316 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" event={"ID":"1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b","Type":"ContainerStarted","Data":"baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4"} Jul 24 12:30:03.356623 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:03.356579 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" event={"ID":"1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b","Type":"ContainerStarted","Data":"445818e42b396d6b3fd49dbc0e0fedd6b2090d07c95722de44299d5253b40261"} Jul 24 12:30:03.374026 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:03.373974 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" podStartSLOduration=1.015485065 podStartE2EDuration="3.373955415s" totalImagesPullingTime="2.35847035s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:30:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:30:03.371422678 +0000 UTC m=+659.656968364" watchObservedRunningTime="2026-07-24 12:30:03.373955415 +0000 UTC m=+659.659501099" Jul 24 12:30:04.363115 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:04.363081 2578 generic.go:356] "Generic (PLEG): container finished" podID="1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b" containerID="445818e42b396d6b3fd49dbc0e0fedd6b2090d07c95722de44299d5253b40261" exitCode=0 Jul 24 12:30:04.363526 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:04.363139 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" event={"ID":"1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b","Type":"ContainerDied","Data":"445818e42b396d6b3fd49dbc0e0fedd6b2090d07c95722de44299d5253b40261"} Jul 24 12:30:05.500894 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:05.500870 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" Jul 24 12:30:06.374448 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:06.374408 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" event={"ID":"1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b","Type":"ContainerDied","Data":"baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4"} Jul 24 12:30:06.374608 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:06.374466 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="baa39486cdc4989c0a298fdcf68fadc93a40e25cd80b4a4d830496539dca74c4" Jul 24 12:30:06.374608 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:06.374427 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748270-6qvqd" Jul 24 12:30:09.982279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.982242 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7996f87bb5-kgxxn"] Jul 24 12:30:09.982663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.982581 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b" containerName="cleanup" Jul 24 12:30:09.982663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.982592 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b" containerName="cleanup" Jul 24 12:30:09.982663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.982646 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b" containerName="cleanup" Jul 24 12:30:09.985650 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.985634 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:09.992933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:09.992903 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7996f87bb5-kgxxn"] Jul 24 12:30:10.080499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.080457 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2f3d971-6797-4dff-a3aa-6c362be25d13-tls-cert\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.080499 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.080496 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zh2hm\" (UniqueName: \"kubernetes.io/projected/d2f3d971-6797-4dff-a3aa-6c362be25d13-kube-api-access-zh2hm\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.181836 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.181796 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2f3d971-6797-4dff-a3aa-6c362be25d13-tls-cert\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.181836 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.181834 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zh2hm\" (UniqueName: \"kubernetes.io/projected/d2f3d971-6797-4dff-a3aa-6c362be25d13-kube-api-access-zh2hm\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.184458 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.184434 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/d2f3d971-6797-4dff-a3aa-6c362be25d13-tls-cert\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.191402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.191381 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zh2hm\" (UniqueName: \"kubernetes.io/projected/d2f3d971-6797-4dff-a3aa-6c362be25d13-kube-api-access-zh2hm\") pod \"authorino-7996f87bb5-kgxxn\" (UID: \"d2f3d971-6797-4dff-a3aa-6c362be25d13\") " pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.295471 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.295439 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7996f87bb5-kgxxn" Jul 24 12:30:10.428032 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:10.428007 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7996f87bb5-kgxxn"] Jul 24 12:30:10.429774 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:30:10.429746 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd2f3d971_6797_4dff_a3aa_6c362be25d13.slice/crio-0c7296d1a046ab620c4b37f1599791fd80f87bdb5e6370d83d4cffb01cfcf1c1 WatchSource:0}: Error finding container 0c7296d1a046ab620c4b37f1599791fd80f87bdb5e6370d83d4cffb01cfcf1c1: Status 404 returned error can't find the container with id 0c7296d1a046ab620c4b37f1599791fd80f87bdb5e6370d83d4cffb01cfcf1c1 Jul 24 12:30:11.403971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.403936 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7996f87bb5-kgxxn" event={"ID":"d2f3d971-6797-4dff-a3aa-6c362be25d13","Type":"ContainerStarted","Data":"4c37cb2797acdabe382f9ddc1e3f6b83b35fcd10bffa1c4bbcc805e9843fdf55"} Jul 24 12:30:11.403971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.403979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7996f87bb5-kgxxn" event={"ID":"d2f3d971-6797-4dff-a3aa-6c362be25d13","Type":"ContainerStarted","Data":"0c7296d1a046ab620c4b37f1599791fd80f87bdb5e6370d83d4cffb01cfcf1c1"} Jul 24 12:30:11.423016 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.422954 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7996f87bb5-kgxxn" podStartSLOduration=1.99397016 podStartE2EDuration="2.422936903s" totalImagesPullingTime="428.966743ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:30:11.422238296 +0000 UTC m=+667.707783982" watchObservedRunningTime="2026-07-24 12:30:11.422936903 +0000 UTC m=+667.708482640" Jul 24 12:30:11.463418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.461146 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:30:11.463885 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.463845 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" podUID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" containerName="authorino" containerID="cri-o://f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95" gracePeriod=30 Jul 24 12:30:11.739532 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.739500 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:30:11.898924 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.898886 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rbb4j\" (UniqueName: \"kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j\") pod \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " Jul 24 12:30:11.899074 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.899023 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert\") pod \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\" (UID: \"9305d35d-b35b-4f4e-a6ce-1dff21a355b1\") " Jul 24 12:30:11.901331 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.901298 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j" (OuterVolumeSpecName: "kube-api-access-rbb4j") pod "9305d35d-b35b-4f4e-a6ce-1dff21a355b1" (UID: "9305d35d-b35b-4f4e-a6ce-1dff21a355b1"). InnerVolumeSpecName "kube-api-access-rbb4j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:30:11.910236 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:11.910206 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "9305d35d-b35b-4f4e-a6ce-1dff21a355b1" (UID: "9305d35d-b35b-4f4e-a6ce-1dff21a355b1"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:30:12.000244 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.000141 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rbb4j\" (UniqueName: \"kubernetes.io/projected/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-kube-api-access-rbb4j\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:30:12.000244 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.000174 2578 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/9305d35d-b35b-4f4e-a6ce-1dff21a355b1-tls-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:30:12.410348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.410306 2578 generic.go:356] "Generic (PLEG): container finished" podID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" containerID="f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95" exitCode=0 Jul 24 12:30:12.410348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.410349 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" Jul 24 12:30:12.410827 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.410385 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" event={"ID":"9305d35d-b35b-4f4e-a6ce-1dff21a355b1","Type":"ContainerDied","Data":"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95"} Jul 24 12:30:12.410827 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.410445 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6dd767fcf6-szcjr" event={"ID":"9305d35d-b35b-4f4e-a6ce-1dff21a355b1","Type":"ContainerDied","Data":"111872482adf703f967e7e66275153ca87eefd4adec96fa0d812270d3da3e845"} Jul 24 12:30:12.410827 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.410468 2578 scope.go:117] "RemoveContainer" containerID="f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95" Jul 24 12:30:12.419625 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.419595 2578 scope.go:117] "RemoveContainer" containerID="f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95" Jul 24 12:30:12.419944 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:30:12.419924 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95\": container with ID starting with f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95 not found: ID does not exist" containerID="f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95" Jul 24 12:30:12.420005 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.419953 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95"} err="failed to get container status \"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95\": rpc error: code = NotFound desc = could not find container \"f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95\": container with ID starting with f3b6a9b14aa8321c6a9b0a5532ecec1b2f77b11aa872abfc08337f7c5513ba95 not found: ID does not exist" Jul 24 12:30:12.426841 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.426812 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:30:12.430808 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:12.430781 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6dd767fcf6-szcjr"] Jul 24 12:30:14.148017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:30:14.147981 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" path="/var/lib/kubelet/pods/9305d35d-b35b-4f4e-a6ce-1dff21a355b1/volumes" Jul 24 12:40:48.599118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.599083 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:40:48.601587 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.599436 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" containerName="authorino" Jul 24 12:40:48.601587 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.599447 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" containerName="authorino" Jul 24 12:40:48.601587 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.599515 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9305d35d-b35b-4f4e-a6ce-1dff21a355b1" containerName="authorino" Jul 24 12:40:48.602437 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.602421 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.605687 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.605659 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-e2f4d877-kserve-self-signed-certs\"" Jul 24 12:40:48.631119 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.631079 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:40:48.729450 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729406 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.729629 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729462 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.729629 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729503 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.729629 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729584 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.729793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729626 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbf7r\" (UniqueName: \"kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.729793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.729698 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.830901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.830856 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.830901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.830903 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbf7r\" (UniqueName: \"kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.830940 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.830994 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.831136 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831356 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.831183 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831471 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.831449 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831577 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.831553 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.831640 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.831582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.833583 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.833554 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.833699 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.833689 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.838713 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.838687 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbf7r\" (UniqueName: \"kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r\") pod \"e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:48.912540 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:48.912430 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:49.080027 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:49.079997 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:40:49.706708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:49.706670 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" event={"ID":"6ccddbe8-9b6f-4647-95a5-74d01478b83b","Type":"ContainerStarted","Data":"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141"} Jul 24 12:40:49.707098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:49.706715 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" event={"ID":"6ccddbe8-9b6f-4647-95a5-74d01478b83b","Type":"ContainerStarted","Data":"cb08640b975ecb13dfee7b98df1933a9a8edb9138c4165b078eaf3f8391fd8bc"} Jul 24 12:40:49.725040 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:49.724992 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" podStartSLOduration=1.724975616 podStartE2EDuration="1.724975616s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:40:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:40:49.7241783 +0000 UTC m=+1306.009723986" watchObservedRunningTime="2026-07-24 12:40:49.724975616 +0000 UTC m=+1306.010521303" Jul 24 12:40:58.912637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:58.912537 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:58.912637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:58.912601 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:58.925529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:58.925497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:40:59.770539 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:40:59.770510 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:41:45.198909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:41:45.198823 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:42:00.218644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.218598 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" podUID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" containerName="main" containerID="cri-o://524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141" gracePeriod=45 Jul 24 12:42:00.473847 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.473788 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:42:00.575110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575070 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575114 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575140 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575184 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575256 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575379 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575284 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fbf7r\" (UniqueName: \"kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r\") pod \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\" (UID: \"6ccddbe8-9b6f-4647-95a5-74d01478b83b\") " Jul 24 12:42:00.575716 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575407 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache" (OuterVolumeSpecName: "model-cache") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:42:00.575716 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575456 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home" (OuterVolumeSpecName: "home") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:42:00.575855 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575832 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-model-cache\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:00.575996 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575863 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-home\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:00.575996 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.575959 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:42:00.578464 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.578420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r" (OuterVolumeSpecName: "kube-api-access-fbf7r") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "kube-api-access-fbf7r". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:42:00.583651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.583627 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:42:00.583750 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.583648 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm" (OuterVolumeSpecName: "dshm") pod "6ccddbe8-9b6f-4647-95a5-74d01478b83b" (UID: "6ccddbe8-9b6f-4647-95a5-74d01478b83b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:42:00.677300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.677265 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-dshm\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:00.677300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.677294 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tmp-dir\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:00.677300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.677304 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6ccddbe8-9b6f-4647-95a5-74d01478b83b-tls-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:00.677529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:00.677315 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fbf7r\" (UniqueName: \"kubernetes.io/projected/6ccddbe8-9b6f-4647-95a5-74d01478b83b-kube-api-access-fbf7r\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:42:01.075838 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.075793 2578 generic.go:356] "Generic (PLEG): container finished" podID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" containerID="524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141" exitCode=0 Jul 24 12:42:01.076037 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.075853 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" Jul 24 12:42:01.076037 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.075878 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" event={"ID":"6ccddbe8-9b6f-4647-95a5-74d01478b83b","Type":"ContainerDied","Data":"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141"} Jul 24 12:42:01.076037 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.075924 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp" event={"ID":"6ccddbe8-9b6f-4647-95a5-74d01478b83b","Type":"ContainerDied","Data":"cb08640b975ecb13dfee7b98df1933a9a8edb9138c4165b078eaf3f8391fd8bc"} Jul 24 12:42:01.076037 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.075942 2578 scope.go:117] "RemoveContainer" containerID="524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141" Jul 24 12:42:01.086078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.086046 2578 scope.go:117] "RemoveContainer" containerID="524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141" Jul 24 12:42:01.086409 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:42:01.086390 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141\": container with ID starting with 524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141 not found: ID does not exist" containerID="524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141" Jul 24 12:42:01.086465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.086419 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141"} err="failed to get container status \"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141\": rpc error: code = NotFound desc = could not find container \"524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141\": container with ID starting with 524bf37f9eb1d2d7c4b199acee414237d8d78efcb0534b3000f1ceb91fe1b141 not found: ID does not exist" Jul 24 12:42:01.098910 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.098884 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:42:01.104460 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:01.104438 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-e2f4d877-kserve-684665779f-zlrsp"] Jul 24 12:42:02.148214 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:42:02.148170 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" path="/var/lib/kubelet/pods/6ccddbe8-9b6f-4647-95a5-74d01478b83b/volumes" Jul 24 12:45:00.133124 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.133043 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh"] Jul 24 12:45:00.133651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.133376 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" containerName="main" Jul 24 12:45:00.133651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.133387 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" containerName="main" Jul 24 12:45:00.133651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.133456 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="6ccddbe8-9b6f-4647-95a5-74d01478b83b" containerName="main" Jul 24 12:45:00.136459 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.136442 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" Jul 24 12:45:00.138295 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.138278 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zw8hg\"" Jul 24 12:45:00.146411 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.146388 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" Jul 24 12:45:00.149851 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.149549 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh"] Jul 24 12:45:00.286404 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:00.286378 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh"] Jul 24 12:45:00.289102 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:45:00.289077 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice/crio-0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9 WatchSource:0}: Error finding container 0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9: Status 404 returned error can't find the container with id 0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9 Jul 24 12:45:01.008316 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:01.008262 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" event={"ID":"e5e5ab11-5307-4580-b385-4b5a6256c66d","Type":"ContainerStarted","Data":"9eb547158c0ff6862430aa012549f25d843f0dc55a803fd892208a41a0e97438"} Jul 24 12:45:01.008316 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:01.008331 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" event={"ID":"e5e5ab11-5307-4580-b385-4b5a6256c66d","Type":"ContainerStarted","Data":"0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9"} Jul 24 12:45:01.022984 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:01.022933 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" podStartSLOduration=1.02291914 podStartE2EDuration="1.02291914s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:45:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:45:01.022033045 +0000 UTC m=+1557.307578732" watchObservedRunningTime="2026-07-24 12:45:01.02291914 +0000 UTC m=+1557.308464826" Jul 24 12:45:02.015116 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:02.015082 2578 generic.go:356] "Generic (PLEG): container finished" podID="e5e5ab11-5307-4580-b385-4b5a6256c66d" containerID="9eb547158c0ff6862430aa012549f25d843f0dc55a803fd892208a41a0e97438" exitCode=0 Jul 24 12:45:02.015526 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:02.015170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" event={"ID":"e5e5ab11-5307-4580-b385-4b5a6256c66d","Type":"ContainerDied","Data":"9eb547158c0ff6862430aa012549f25d843f0dc55a803fd892208a41a0e97438"} Jul 24 12:45:03.159740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:03.159714 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" Jul 24 12:45:04.028028 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:04.027993 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" event={"ID":"e5e5ab11-5307-4580-b385-4b5a6256c66d","Type":"ContainerDied","Data":"0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9"} Jul 24 12:45:04.028184 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:04.028043 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0cb0dbaf6064a9ca9e93690816e26178f76a1a2c03ed03bf0e7540dbd882cbb9" Jul 24 12:45:04.028184 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:04.027998 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29748285-mjlvh" Jul 24 12:45:06.711539 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:06.711498 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:06.712226 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:06.711515 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:08.344510 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:08.344474 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:14.859980 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:14.859941 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:18.353873 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:18.353842 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:28.362475 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:28.362441 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:29.821932 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:29.821893 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:30.597410 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.597374 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:45:30.597730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.597716 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5e5ab11-5307-4580-b385-4b5a6256c66d" containerName="cleanup" Jul 24 12:45:30.597730 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.597730 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5e5ab11-5307-4580-b385-4b5a6256c66d" containerName="cleanup" Jul 24 12:45:30.597840 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.597786 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5e5ab11-5307-4580-b385-4b5a6256c66d" containerName="cleanup" Jul 24 12:45:30.600986 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.600964 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.605323 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.605301 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-227c45ac-openshift-default-dockercfg-tg24s\"" Jul 24 12:45:30.612450 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.612427 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:45:30.650503 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650670 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650511 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5h4td\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650670 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650535 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650670 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650611 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650779 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650779 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650779 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650748 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650872 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650784 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.650872 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.650830 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751264 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751222 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751289 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751323 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5h4td\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751389 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751415 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751467 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751441 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751494 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751793 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751547 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751966 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751914 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.751966 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.751948 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.752110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.752051 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.752173 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.752143 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.752251 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.752177 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.754632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.754603 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.754792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.754776 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.758828 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.758801 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.759104 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.759082 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5h4td\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td\") pod \"e2e-ait-227c45ac-openshift-default-85b974679d-lm75q\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:30.913160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:30.913062 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:31.057440 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.057415 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:45:31.059476 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:45:31.059446 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5fbe4321_2356_4439_86f6_2a262ff77b80.slice/crio-9c00eb31a006fb27601b7a9c891c894df5a470f848c315e5a6ae04c942e5fcde WatchSource:0}: Error finding container 9c00eb31a006fb27601b7a9c891c894df5a470f848c315e5a6ae04c942e5fcde: Status 404 returned error can't find the container with id 9c00eb31a006fb27601b7a9c891c894df5a470f848c315e5a6ae04c942e5fcde Jul 24 12:45:31.061651 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.061616 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:31.061753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.061688 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:31.061753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.061716 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:31.175616 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.175514 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerStarted","Data":"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59"} Jul 24 12:45:31.175616 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.175581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerStarted","Data":"9c00eb31a006fb27601b7a9c891c894df5a470f848c315e5a6ae04c942e5fcde"} Jul 24 12:45:31.210950 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.210897 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podStartSLOduration=1.210879288 podStartE2EDuration="1.210879288s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:45:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:45:31.208266124 +0000 UTC m=+1587.493811811" watchObservedRunningTime="2026-07-24 12:45:31.210879288 +0000 UTC m=+1587.496425029" Jul 24 12:45:31.914232 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:31.914156 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:32.919387 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:32.919351 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:33.185804 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:33.185720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:33.186792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:33.186768 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:38.406015 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:38.405980 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:42.819306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.819250 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn"] Jul 24 12:45:42.823368 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.823340 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:42.825476 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.825450 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 12:45:42.825609 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.825456 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zw8hg\"" Jul 24 12:45:42.830803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.830674 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn"] Jul 24 12:45:42.965206 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:42.965165 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn\" (UID: \"1747b05c-dc8d-4876-8c3d-0ef247c7b994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:43.066229 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:43.066178 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn\" (UID: \"1747b05c-dc8d-4876-8c3d-0ef247c7b994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:43.066866 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:43.066838 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn\" (UID: \"1747b05c-dc8d-4876-8c3d-0ef247c7b994\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:43.137433 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:43.137330 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:43.297318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:43.297289 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn"] Jul 24 12:45:44.244219 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:44.244153 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" event={"ID":"1747b05c-dc8d-4876-8c3d-0ef247c7b994","Type":"ContainerStarted","Data":"3c08da9b9c7529e38533667cf18b20e3f445c3c258e36d09526875bf7eb43c70"} Jul 24 12:45:44.244618 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:44.244246 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" event={"ID":"1747b05c-dc8d-4876-8c3d-0ef247c7b994","Type":"ContainerStarted","Data":"10eff247da72440ca2c0bdfaade035d3b8c34530a88947b501f5c19740d75ca5"} Jul 24 12:45:44.261110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:44.261063 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" podStartSLOduration=2.2610420700000002 podStartE2EDuration="2.26104207s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:45:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:45:44.259661039 +0000 UTC m=+1600.545206730" watchObservedRunningTime="2026-07-24 12:45:44.26104207 +0000 UTC m=+1600.546587764" Jul 24 12:45:44.853242 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:44.853206 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:45.251815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:45.251773 2578 generic.go:356] "Generic (PLEG): container finished" podID="1747b05c-dc8d-4876-8c3d-0ef247c7b994" containerID="3c08da9b9c7529e38533667cf18b20e3f445c3c258e36d09526875bf7eb43c70" exitCode=0 Jul 24 12:45:45.252179 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:45.251862 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" event={"ID":"1747b05c-dc8d-4876-8c3d-0ef247c7b994","Type":"ContainerDied","Data":"3c08da9b9c7529e38533667cf18b20e3f445c3c258e36d09526875bf7eb43c70"} Jul 24 12:45:46.398960 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:46.398929 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:46.498610 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:46.498565 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca\") pod \"1747b05c-dc8d-4876-8c3d-0ef247c7b994\" (UID: \"1747b05c-dc8d-4876-8c3d-0ef247c7b994\") " Jul 24 12:45:46.498951 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:46.498930 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1747b05c-dc8d-4876-8c3d-0ef247c7b994" (UID: "1747b05c-dc8d-4876-8c3d-0ef247c7b994"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:45:46.599554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:46.599502 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1747b05c-dc8d-4876-8c3d-0ef247c7b994-maas-api-service-ca\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:45:47.265375 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:47.265331 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" Jul 24 12:45:47.265375 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:47.265366 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn" event={"ID":"1747b05c-dc8d-4876-8c3d-0ef247c7b994","Type":"ContainerDied","Data":"10eff247da72440ca2c0bdfaade035d3b8c34530a88947b501f5c19740d75ca5"} Jul 24 12:45:47.265647 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:47.265417 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="10eff247da72440ca2c0bdfaade035d3b8c34530a88947b501f5c19740d75ca5" Jul 24 12:45:48.187392 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.187309 2578 patch_prober.go:28] interesting pod/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 12:45:48.187392 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.187364 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 12:45:48.272023 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.271996 2578 generic.go:356] "Generic (PLEG): container finished" podID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerID="23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59" exitCode=0 Jul 24 12:45:48.272181 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.272045 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerDied","Data":"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59"} Jul 24 12:45:48.272465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.272450 2578 scope.go:117] "RemoveContainer" containerID="23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59" Jul 24 12:45:48.273440 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.273413 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:48.273498 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.273484 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:48.273541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:48.273529 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:48.415151 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:48.415118 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:49.278987 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:49.278953 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerStarted","Data":"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647"} Jul 24 12:45:49.914091 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:49.914053 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:49.914371 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:49.914340 2578 patch_prober.go:28] interesting pod/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 12:45:49.914585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:49.914558 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 12:45:50.914130 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:50.914084 2578 patch_prober.go:28] interesting pod/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 12:45:50.914778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:50.914742 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 12:45:51.914214 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:51.914147 2578 patch_prober.go:28] interesting pod/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 12:45:51.914835 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:51.914800 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 12:45:52.914291 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:52.914250 2578 patch_prober.go:28] interesting pod/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" start-of-body= Jul 24 12:45:52.914922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:52.914887 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.52:15021/healthz/ready\": dial tcp 10.133.0.52:15021: connect: connection refused" Jul 24 12:45:53.918242 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:53.918186 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:53.918630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:53.918553 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:53.919374 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:53.919353 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:45:57.908514 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:57.908405 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn"] Jul 24 12:45:57.911291 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:57.911259 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-227c45ac-7pfcn"] Jul 24 12:45:58.044953 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:58.044919 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:45:58.045168 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:58.045146 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" containerID="cri-o://80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647" gracePeriod=30 Jul 24 12:45:58.151385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:58.151341 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1747b05c-dc8d-4876-8c3d-0ef247c7b994" path="/var/lib/kubelet/pods/1747b05c-dc8d-4876-8c3d-0ef247c7b994/volumes" Jul 24 12:45:58.459286 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:58.459252 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:45:59.085672 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.085641 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:45:59.086131 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.085981 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1747b05c-dc8d-4876-8c3d-0ef247c7b994" containerName="revoke-keys" Jul 24 12:45:59.086131 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.085993 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1747b05c-dc8d-4876-8c3d-0ef247c7b994" containerName="revoke-keys" Jul 24 12:45:59.086131 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.086057 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1747b05c-dc8d-4876-8c3d-0ef247c7b994" containerName="revoke-keys" Jul 24 12:45:59.089866 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.089844 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.092579 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.092558 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-b0a74db1-gw-openshift-default-dockercfg-xdfwj\"" Jul 24 12:45:59.103236 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.103206 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:45:59.217804 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.217769 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.217828 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.217887 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.217938 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.217993 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6p885\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.218038 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.218062 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.218101 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.218315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.218134 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.318860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318816 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6p885\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.318860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318866 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318884 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318915 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318938 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.318967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319003 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319034 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319087 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319566 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319487 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319566 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319666 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319646 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319749 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319710 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.319858 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.319833 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.321555 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.321530 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.321892 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.321864 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.327157 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.327130 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.327377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.327359 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6p885\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885\") pod \"e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.400419 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.400329 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:45:59.547000 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.546974 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:45:59.549828 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:45:59.549798 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0cf2832c_0cb7_46c6_a1b0_b5eeb3c4efaf.slice/crio-2c9a17f3e7f252e637fad3bc0d1f4e5e70e3c1e2cf3f3f8c5c38041bd7e5c289 WatchSource:0}: Error finding container 2c9a17f3e7f252e637fad3bc0d1f4e5e70e3c1e2cf3f3f8c5c38041bd7e5c289: Status 404 returned error can't find the container with id 2c9a17f3e7f252e637fad3bc0d1f4e5e70e3c1e2cf3f3f8c5c38041bd7e5c289 Jul 24 12:45:59.552125 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.552082 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:59.552284 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.552263 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:59.552359 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:45:59.552317 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:45:59.818874 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:45:59.818831 2578 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5e5ab11_5307_4580_b385_4b5a6256c66d.slice\": RecentStats: unable to find data in memory cache]" Jul 24 12:46:00.344719 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:00.344683 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerStarted","Data":"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d"} Jul 24 12:46:00.345130 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:00.344764 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerStarted","Data":"2c9a17f3e7f252e637fad3bc0d1f4e5e70e3c1e2cf3f3f8c5c38041bd7e5c289"} Jul 24 12:46:00.367783 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:00.367723 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podStartSLOduration=1.367707075 podStartE2EDuration="1.367707075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:45:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:46:00.365585935 +0000 UTC m=+1616.651131637" watchObservedRunningTime="2026-07-24 12:46:00.367707075 +0000 UTC m=+1616.653252761" Jul 24 12:46:00.401014 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:00.400979 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:00.406308 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:00.406282 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:01.350232 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:01.350187 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:01.351277 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:01.351257 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:03.301902 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.301868 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:46:03.363883 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.363807 2578 generic.go:356] "Generic (PLEG): container finished" podID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerID="80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647" exitCode=0 Jul 24 12:46:03.363883 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.363858 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" Jul 24 12:46:03.364048 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.363887 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerDied","Data":"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647"} Jul 24 12:46:03.364048 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.363949 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q" event={"ID":"5fbe4321-2356-4439-86f6-2a262ff77b80","Type":"ContainerDied","Data":"9c00eb31a006fb27601b7a9c891c894df5a470f848c315e5a6ae04c942e5fcde"} Jul 24 12:46:03.364048 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.363980 2578 scope.go:117] "RemoveContainer" containerID="80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647" Jul 24 12:46:03.375247 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.375208 2578 scope.go:117] "RemoveContainer" containerID="23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59" Jul 24 12:46:03.383339 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.383314 2578 scope.go:117] "RemoveContainer" containerID="80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647" Jul 24 12:46:03.383597 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:46:03.383577 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647\": container with ID starting with 80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647 not found: ID does not exist" containerID="80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647" Jul 24 12:46:03.383649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.383608 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647"} err="failed to get container status \"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647\": rpc error: code = NotFound desc = could not find container \"80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647\": container with ID starting with 80f910ef6ecafc1d34de1af7d72ae0f616be3de580689d1b22b434597f251647 not found: ID does not exist" Jul 24 12:46:03.383649 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.383627 2578 scope.go:117] "RemoveContainer" containerID="23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59" Jul 24 12:46:03.383901 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:46:03.383883 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59\": container with ID starting with 23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59 not found: ID does not exist" containerID="23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59" Jul 24 12:46:03.383943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.383908 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59"} err="failed to get container status \"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59\": rpc error: code = NotFound desc = could not find container \"23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59\": container with ID starting with 23671045ed7d5094be93ba2abb276659b7fd6919c378f9ab83bc13f13c220f59 not found: ID does not exist" Jul 24 12:46:03.457870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.457832 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.457870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.457871 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.457893 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.457972 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5h4td\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458001 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458024 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458064 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458093 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458117 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs\") pod \"5fbe4321-2356-4439-86f6-2a262ff77b80\" (UID: \"5fbe4321-2356-4439-86f6-2a262ff77b80\") " Jul 24 12:46:03.458444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458234 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:03.458444 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:03.458611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458458 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:03.458673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458617 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.458673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458637 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.458673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458652 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.458805 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458694 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:46:03.458936 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.458906 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data" (OuterVolumeSpecName: "istio-data") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:03.460492 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.460469 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:46:03.460612 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.460538 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token" (OuterVolumeSpecName: "istio-token") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:46:03.460612 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.460604 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:03.460852 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.460831 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td" (OuterVolumeSpecName: "kube-api-access-5h4td") pod "5fbe4321-2356-4439-86f6-2a262ff77b80" (UID: "5fbe4321-2356-4439-86f6-2a262ff77b80"). InnerVolumeSpecName "kube-api-access-5h4td". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:46:03.560017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.559977 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5h4td\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-kube-api-access-5h4td\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.560017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.560011 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.560017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.560022 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.560017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.560030 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5fbe4321-2356-4439-86f6-2a262ff77b80-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.560344 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.560037 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.560344 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.560046 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5fbe4321-2356-4439-86f6-2a262ff77b80-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:03.687288 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.687253 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:46:03.692050 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:03.692017 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-227c45ac-openshift-default-85b974679d-lm75q"] Jul 24 12:46:04.150241 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:04.150209 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" path="/var/lib/kubelet/pods/5fbe4321-2356-4439-86f6-2a262ff77b80/volumes" Jul 24 12:46:15.436044 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.435948 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerID="867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d" exitCode=0 Jul 24 12:46:15.436044 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.436026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerDied","Data":"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d"} Jul 24 12:46:15.436541 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.436448 2578 scope.go:117] "RemoveContainer" containerID="867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d" Jul 24 12:46:15.437490 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.437462 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:15.437562 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.437533 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:15.437598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:15.437565 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:16.351090 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:16.351049 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:16.443169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:16.443133 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerStarted","Data":"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d"} Jul 24 12:46:17.401284 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:17.401248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:17.402682 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:17.402649 2578 patch_prober.go:28] interesting pod/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 12:46:17.402820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:17.402717 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 12:46:18.401486 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:18.401441 2578 patch_prober.go:28] interesting pod/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 12:46:18.402014 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:18.401516 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 12:46:19.401757 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:19.401713 2578 patch_prober.go:28] interesting pod/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 12:46:19.402417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:19.402372 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 12:46:20.401599 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:20.401559 2578 patch_prober.go:28] interesting pod/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" start-of-body= Jul 24 12:46:20.401813 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:20.401615 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.54:15021/healthz/ready\": dial tcp 10.133.0.54:15021: connect: connection refused" Jul 24 12:46:21.405423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:21.405384 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:21.405903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:21.405765 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:21.406334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:21.406314 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:42.192784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:42.192741 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:46:42.193399 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:42.192981 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" containerID="cri-o://7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d" gracePeriod=30 Jul 24 12:46:43.034545 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034482 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:46:43.034860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034848 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.034905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034862 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.034905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034870 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.034905 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034875 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.034999 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034940 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.034999 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.034948 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5fbe4321-2356-4439-86f6-2a262ff77b80" containerName="istio-proxy" Jul 24 12:46:43.038114 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.038077 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.040763 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.040739 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-52688321-openshift-default-dockercfg-smbpd\"" Jul 24 12:46:43.052772 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.052748 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:46:43.142080 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142040 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142080 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142080 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142157 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142228 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcnzn\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142252 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142292 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142339 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142365 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.142497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.142397 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243110 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243065 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243117 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243145 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243264 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243292 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243326 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243355 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcnzn\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243568 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243582 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.243628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243595 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.244118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243734 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.244118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243978 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.244118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.243993 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.246026 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.246001 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.246298 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.246277 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.256376 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.256354 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.256570 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.256550 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcnzn\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn\") pod \"e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.348778 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.348700 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:43.505019 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.504983 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:46:43.512901 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:46:43.512861 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod988701d5_710a_46b4_8467_41cad4eded9e.slice/crio-0fbf3ebe584a6caadc5cc6cc3466eaeb01dc4929312bea6e0ece62e68ad6d839 WatchSource:0}: Error finding container 0fbf3ebe584a6caadc5cc6cc3466eaeb01dc4929312bea6e0ece62e68ad6d839: Status 404 returned error can't find the container with id 0fbf3ebe584a6caadc5cc6cc3466eaeb01dc4929312bea6e0ece62e68ad6d839 Jul 24 12:46:43.514933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.514900 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:43.515038 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.514982 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:43.515038 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.515027 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:43.595365 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.595327 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerStarted","Data":"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39"} Jul 24 12:46:43.595536 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.595391 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerStarted","Data":"0fbf3ebe584a6caadc5cc6cc3466eaeb01dc4929312bea6e0ece62e68ad6d839"} Jul 24 12:46:43.621610 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:43.621508 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podStartSLOduration=0.621489545 podStartE2EDuration="621.489545ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:46:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:46:43.620273189 +0000 UTC m=+1659.905818896" watchObservedRunningTime="2026-07-24 12:46:43.621489545 +0000 UTC m=+1659.907035291" Jul 24 12:46:44.349365 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:44.349329 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:44.359729 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:44.355042 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:44.600595 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:44.600510 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:44.601421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:44.601399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:46:47.453558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.453536 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:47.483366 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483275 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483366 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483347 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483402 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483428 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6p885\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483637 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483587 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483669 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483694 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483762 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data\") pod \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\" (UID: \"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf\") " Jul 24 12:46:47.483843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.483794 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:46:47.484056 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.484034 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.484275 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.484186 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:47.484747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.484652 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:47.484851 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.484760 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:47.484851 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.484832 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data" (OuterVolumeSpecName: "istio-data") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:47.487521 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.487479 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token" (OuterVolumeSpecName: "istio-token") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:46:47.487521 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.487478 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885" (OuterVolumeSpecName: "kube-api-access-6p885") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "kube-api-access-6p885". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:46:47.487696 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.487576 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:46:47.487904 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.487884 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" (UID: "0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:46:47.585494 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585460 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585494 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585488 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585494 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585500 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6p885\" (UniqueName: \"kubernetes.io/projected/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-kube-api-access-6p885\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585510 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585518 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585526 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585534 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.585737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.585543 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:46:47.618324 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.618287 2578 generic.go:356] "Generic (PLEG): container finished" podID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerID="7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d" exitCode=0 Jul 24 12:46:47.618517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.618341 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" Jul 24 12:46:47.618517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.618371 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerDied","Data":"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d"} Jul 24 12:46:47.618517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.618445 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd" event={"ID":"0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf","Type":"ContainerDied","Data":"2c9a17f3e7f252e637fad3bc0d1f4e5e70e3c1e2cf3f3f8c5c38041bd7e5c289"} Jul 24 12:46:47.618517 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.618476 2578 scope.go:117] "RemoveContainer" containerID="7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d" Jul 24 12:46:47.628116 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.628096 2578 scope.go:117] "RemoveContainer" containerID="867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d" Jul 24 12:46:47.636358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.636341 2578 scope.go:117] "RemoveContainer" containerID="7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d" Jul 24 12:46:47.636635 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:46:47.636616 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d\": container with ID starting with 7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d not found: ID does not exist" containerID="7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d" Jul 24 12:46:47.636685 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.636645 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d"} err="failed to get container status \"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d\": rpc error: code = NotFound desc = could not find container \"7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d\": container with ID starting with 7d402a80fcb21d9c7c1b0b16f94731b7dbb95a566a65b2177b626bb215980d9d not found: ID does not exist" Jul 24 12:46:47.636685 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.636665 2578 scope.go:117] "RemoveContainer" containerID="867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d" Jul 24 12:46:47.636926 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:46:47.636906 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d\": container with ID starting with 867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d not found: ID does not exist" containerID="867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d" Jul 24 12:46:47.637027 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.636929 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d"} err="failed to get container status \"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d\": rpc error: code = NotFound desc = could not find container \"867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d\": container with ID starting with 867e1158ba9526bbe9742433c536e028af189745d871ff8410f4c39926368e9d not found: ID does not exist" Jul 24 12:46:47.640900 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.640876 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:46:47.644205 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:47.644168 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-b0a74db1-gw-openshift-default-69dffb6bb4-tckpd"] Jul 24 12:46:48.149265 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:48.149234 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" path="/var/lib/kubelet/pods/0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf/volumes" Jul 24 12:46:59.602648 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.602608 2578 patch_prober.go:28] interesting pod/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 12:46:59.603071 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.602664 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 12:46:59.689762 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.689678 2578 generic.go:356] "Generic (PLEG): container finished" podID="988701d5-710a-46b4-8467-41cad4eded9e" containerID="d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39" exitCode=0 Jul 24 12:46:59.689762 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.689753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerDied","Data":"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39"} Jul 24 12:46:59.690223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.690186 2578 scope.go:117] "RemoveContainer" containerID="d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39" Jul 24 12:46:59.691244 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.691215 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:59.691315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.691300 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:46:59.691355 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:46:59.691337 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:00.696781 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:00.696748 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerStarted","Data":"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86"} Jul 24 12:47:01.349899 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:01.349861 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:01.350117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:01.350096 2578 patch_prober.go:28] interesting pod/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 12:47:01.350169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:01.350150 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 12:47:02.349246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:02.349167 2578 patch_prober.go:28] interesting pod/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 12:47:02.349675 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:02.349274 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 12:47:03.349722 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:03.349689 2578 patch_prober.go:28] interesting pod/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 12:47:03.350089 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:03.349760 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 12:47:04.349421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:04.349383 2578 patch_prober.go:28] interesting pod/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body= Jul 24 12:47:04.349608 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:04.349452 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" Jul 24 12:47:05.354149 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:05.354119 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:05.354647 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:05.354527 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:05.355383 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:05.355355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:27.533636 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:27.533553 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:47:27.534128 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:27.533821 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" containerID="cri-o://1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86" gracePeriod=30 Jul 24 12:47:28.329322 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329285 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:47:28.329673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329658 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.329739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329678 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.329739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329692 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.329739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329698 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.329947 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329759 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.329947 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.329766 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="0cf2832c-0cb7-46c6-a1b0-b5eeb3c4efaf" containerName="istio-proxy" Jul 24 12:47:28.334085 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.334065 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.336391 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.336365 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-266bd0de-openshift-default-dockercfg-vghfm\"" Jul 24 12:47:28.348409 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.348378 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:47:28.351814 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.351786 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.351814 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.351823 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352026 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352008 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352121 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352105 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352182 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352133 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352255 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352178 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352345 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352312 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.352345 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.352330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwcn9\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453709 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453677 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453709 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453711 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453747 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453783 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453811 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453851 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453892 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453919 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.453969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.453942 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwcn9\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.454349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.454291 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.454349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.454333 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.454587 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.454563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.454767 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.454606 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.454877 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.454612 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.456554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.456535 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.456940 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.456924 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.462597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.462575 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.462708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.462610 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwcn9\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9\") pod \"e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.649296 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.649178 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:28.795750 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.795725 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:47:28.797529 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:47:28.797503 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaee7a03c_345a_49ea_85ce_fa170579e04c.slice/crio-1cb627beb758d669c138830bda690414db25293ce2cfe55c8497a5647853c93c WatchSource:0}: Error finding container 1cb627beb758d669c138830bda690414db25293ce2cfe55c8497a5647853c93c: Status 404 returned error can't find the container with id 1cb627beb758d669c138830bda690414db25293ce2cfe55c8497a5647853c93c Jul 24 12:47:28.800099 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.800063 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:28.800178 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.800131 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:28.800178 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.800171 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:28.856867 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:28.856827 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerStarted","Data":"1cb627beb758d669c138830bda690414db25293ce2cfe55c8497a5647853c93c"} Jul 24 12:47:29.864886 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:29.864849 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerStarted","Data":"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb"} Jul 24 12:47:29.885127 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:29.885075 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podStartSLOduration=1.885059918 podStartE2EDuration="1.885059918s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:47:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:47:29.882893717 +0000 UTC m=+1706.168439402" watchObservedRunningTime="2026-07-24 12:47:29.885059918 +0000 UTC m=+1706.170605603" Jul 24 12:47:30.650247 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:30.650211 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:30.655338 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:30.655309 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:30.870523 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:30.870494 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:30.871524 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:30.871504 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:32.791536 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.791512 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:32.882177 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.882082 2578 generic.go:356] "Generic (PLEG): container finished" podID="988701d5-710a-46b4-8467-41cad4eded9e" containerID="1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86" exitCode=0 Jul 24 12:47:32.882177 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.882130 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" Jul 24 12:47:32.882455 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.882170 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerDied","Data":"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86"} Jul 24 12:47:32.882455 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.882289 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt" event={"ID":"988701d5-710a-46b4-8467-41cad4eded9e","Type":"ContainerDied","Data":"0fbf3ebe584a6caadc5cc6cc3466eaeb01dc4929312bea6e0ece62e68ad6d839"} Jul 24 12:47:32.882455 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.882325 2578 scope.go:117] "RemoveContainer" containerID="1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86" Jul 24 12:47:32.891329 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.891311 2578 scope.go:117] "RemoveContainer" containerID="d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39" Jul 24 12:47:32.894408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894384 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894488 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894418 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894488 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894450 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894590 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894566 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hcnzn\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894665 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894703 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894732 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894755 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.894845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894699 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:47:32.894845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894778 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:47:32.894845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894836 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token\") pod \"988701d5-710a-46b4-8467-41cad4eded9e\" (UID: \"988701d5-710a-46b4-8467-41cad4eded9e\") " Jul 24 12:47:32.895052 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.894920 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:47:32.895342 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.895320 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.895463 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.895345 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/988701d5-710a-46b4-8467-41cad4eded9e-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.895463 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.895345 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:47:32.895463 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.895360 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.895786 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.895762 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data" (OuterVolumeSpecName: "istio-data") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:47:32.896831 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.896806 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:47:32.897273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.897171 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:47:32.897407 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.897389 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn" (OuterVolumeSpecName: "kube-api-access-hcnzn") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "kube-api-access-hcnzn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:47:32.897520 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.897500 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token" (OuterVolumeSpecName: "istio-token") pod "988701d5-710a-46b4-8467-41cad4eded9e" (UID: "988701d5-710a-46b4-8467-41cad4eded9e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:47:32.900601 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.900584 2578 scope.go:117] "RemoveContainer" containerID="1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86" Jul 24 12:47:32.900925 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:47:32.900886 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86\": container with ID starting with 1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86 not found: ID does not exist" containerID="1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86" Jul 24 12:47:32.900998 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.900932 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86"} err="failed to get container status \"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86\": rpc error: code = NotFound desc = could not find container \"1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86\": container with ID starting with 1e117d425274e8fc3c2d1f3126a69060bdefddd1383646997e7209c9985eae86 not found: ID does not exist" Jul 24 12:47:32.900998 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.900950 2578 scope.go:117] "RemoveContainer" containerID="d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39" Jul 24 12:47:32.901242 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:47:32.901226 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39\": container with ID starting with d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39 not found: ID does not exist" containerID="d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39" Jul 24 12:47:32.901307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.901246 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39"} err="failed to get container status \"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39\": rpc error: code = NotFound desc = could not find container \"d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39\": container with ID starting with d8e2d7b1dae2011aefeba82f9313a7856cda959c49b864b32e5b7f1c7efc5b39 not found: ID does not exist" Jul 24 12:47:32.996036 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.995994 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.996036 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.996030 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.996036 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.996044 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/988701d5-710a-46b4-8467-41cad4eded9e-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.996358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.996055 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.996358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.996068 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/988701d5-710a-46b4-8467-41cad4eded9e-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:32.996358 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:32.996080 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hcnzn\" (UniqueName: \"kubernetes.io/projected/988701d5-710a-46b4-8467-41cad4eded9e-kube-api-access-hcnzn\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:47:33.204253 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:33.204218 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:47:33.208475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:33.208449 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-52688321-openshift-default-7c8d7856c4-r58nt"] Jul 24 12:47:34.149481 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:34.149442 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="988701d5-710a-46b4-8467-41cad4eded9e" path="/var/lib/kubelet/pods/988701d5-710a-46b4-8467-41cad4eded9e/volumes" Jul 24 12:47:44.959312 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.959275 2578 generic.go:356] "Generic (PLEG): container finished" podID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerID="08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb" exitCode=0 Jul 24 12:47:44.959758 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.959354 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerDied","Data":"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb"} Jul 24 12:47:44.959845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.959823 2578 scope.go:117] "RemoveContainer" containerID="08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb" Jul 24 12:47:44.960856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.960828 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:44.960915 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.960895 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:44.960951 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:44.960927 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:47:45.870860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:45.870812 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:45.966758 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:45.966726 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerStarted","Data":"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6"} Jul 24 12:47:46.650183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:46.650144 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:46.651511 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:46.651483 2578 patch_prober.go:28] interesting pod/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 12:47:46.651673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:46.651532 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 12:47:47.650268 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:47.650229 2578 patch_prober.go:28] interesting pod/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 12:47:47.650700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:47.650300 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 12:47:48.649899 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:48.649858 2578 patch_prober.go:28] interesting pod/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 12:47:48.650083 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:48.649914 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 12:47:49.649618 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:49.649577 2578 patch_prober.go:28] interesting pod/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" start-of-body= Jul 24 12:47:49.650102 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:49.649640 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.56:15021/healthz/ready\": dial tcp 10.133.0.56:15021: connect: connection refused" Jul 24 12:47:50.654150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:50.654117 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:50.654551 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:50.654404 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:47:50.655028 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:47:50.655007 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:48:11.404421 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:11.404382 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:48:11.404856 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:11.404618 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" containerID="cri-o://7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6" gracePeriod=30 Jul 24 12:48:16.658063 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.658037 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:48:16.681173 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681103 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681173 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681139 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681173 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681169 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681224 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681263 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mwcn9\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681308 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681330 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681377 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681449 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681424 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket\") pod \"aee7a03c-345a-49ea-85ce-fa170579e04c\" (UID: \"aee7a03c-345a-49ea-85ce-fa170579e04c\") " Jul 24 12:48:16.681763 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681429 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:48:16.681763 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681609 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:48:16.681884 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681784 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:48:16.681884 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681801 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:48:16.681995 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.681936 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/aee7a03c-345a-49ea-85ce-fa170579e04c-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.682142 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.682047 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.682377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.682350 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data" (OuterVolumeSpecName: "istio-data") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:48:16.683684 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.683649 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:48:16.683684 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.683665 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9" (OuterVolumeSpecName: "kube-api-access-mwcn9") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "kube-api-access-mwcn9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:48:16.683823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.683737 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token" (OuterVolumeSpecName: "istio-token") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:48:16.683823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.683783 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "aee7a03c-345a-49ea-85ce-fa170579e04c" (UID: "aee7a03c-345a-49ea-85ce-fa170579e04c"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:48:16.783012 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.782969 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783012 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783003 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783012 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783016 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783029 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mwcn9\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-kube-api-access-mwcn9\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783041 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783053 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:16.783311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:16.783063 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/aee7a03c-345a-49ea-85ce-fa170579e04c-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:48:17.133073 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.133037 2578 generic.go:356] "Generic (PLEG): container finished" podID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerID="7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6" exitCode=0 Jul 24 12:48:17.133287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.133090 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" Jul 24 12:48:17.133287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.133102 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerDied","Data":"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6"} Jul 24 12:48:17.133287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.133147 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h" event={"ID":"aee7a03c-345a-49ea-85ce-fa170579e04c","Type":"ContainerDied","Data":"1cb627beb758d669c138830bda690414db25293ce2cfe55c8497a5647853c93c"} Jul 24 12:48:17.133287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.133168 2578 scope.go:117] "RemoveContainer" containerID="7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6" Jul 24 12:48:17.143031 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.143013 2578 scope.go:117] "RemoveContainer" containerID="08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb" Jul 24 12:48:17.151827 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.151809 2578 scope.go:117] "RemoveContainer" containerID="7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6" Jul 24 12:48:17.152131 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:48:17.152110 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6\": container with ID starting with 7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6 not found: ID does not exist" containerID="7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6" Jul 24 12:48:17.152223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.152145 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6"} err="failed to get container status \"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6\": rpc error: code = NotFound desc = could not find container \"7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6\": container with ID starting with 7e3ca7099617b415a25642d40d9ae15bb64f7951b65b91f1837bed5e1e0e36a6 not found: ID does not exist" Jul 24 12:48:17.152223 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.152171 2578 scope.go:117] "RemoveContainer" containerID="08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb" Jul 24 12:48:17.152458 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:48:17.152442 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb\": container with ID starting with 08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb not found: ID does not exist" containerID="08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb" Jul 24 12:48:17.152506 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.152467 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb"} err="failed to get container status \"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb\": rpc error: code = NotFound desc = could not find container \"08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb\": container with ID starting with 08b65cc768246fbdd2091952e67efea472d3bac742dfeeba3ff37bf7d09264cb not found: ID does not exist" Jul 24 12:48:17.156353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.156328 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:48:17.164949 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:17.164925 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-266bd0de-openshift-default-58bbf4d8-jl25h"] Jul 24 12:48:18.148082 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:48:18.148049 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" path="/var/lib/kubelet/pods/aee7a03c-345a-49ea-85ce-fa170579e04c/volumes" Jul 24 12:49:50.910799 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.910761 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911112 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911123 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911134 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911140 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911162 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911169 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911240 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911248 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:49:50.911422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.911255 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="988701d5-710a-46b4-8467-41cad4eded9e" containerName="istio-proxy" Jul 24 12:49:50.914349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.914332 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:50.916507 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.916490 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e38b46e9-openshift-default-dockercfg-cjzxm\"" Jul 24 12:49:50.939855 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:50.939830 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:49:51.030332 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030285 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030544 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030544 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030447 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030544 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030490 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030570 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030614 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030695 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030718 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.030949 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.030774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ddcsv\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131556 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131514 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131587 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131612 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131660 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ddcsv\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131691 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.131753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131723 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131753 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131780 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131818 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132024 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.131983 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132250 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.132084 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132660 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.132630 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132775 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.132643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.132872 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.132851 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.134841 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.134812 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.135069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.135039 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.142552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.142514 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.142751 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.142728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ddcsv\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv\") pod \"e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.225422 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.225323 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:51.382088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.382056 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:49:51.383118 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:49:51.383089 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod44a920c8_1a4b_4e2b_9a1d_0b20a9c53223.slice/crio-19424e1f4c9a60409ecde0ef9b2e8a3917e48109ef8b127f3f43255fddb643bf WatchSource:0}: Error finding container 19424e1f4c9a60409ecde0ef9b2e8a3917e48109ef8b127f3f43255fddb643bf: Status 404 returned error can't find the container with id 19424e1f4c9a60409ecde0ef9b2e8a3917e48109ef8b127f3f43255fddb643bf Jul 24 12:49:51.385951 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.385915 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:49:51.386058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.385991 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:49:51.386058 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.386030 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:49:51.630610 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.630562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerStarted","Data":"d4930d15470c0f2bd684df6f3b7f7af57da691edc2db555fc4c7beda33193761"} Jul 24 12:49:51.630823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.630642 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerStarted","Data":"19424e1f4c9a60409ecde0ef9b2e8a3917e48109ef8b127f3f43255fddb643bf"} Jul 24 12:49:51.652086 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:51.652025 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podStartSLOduration=1.6520026909999999 podStartE2EDuration="1.652002691s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:49:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:49:51.651017557 +0000 UTC m=+1847.936563265" watchObservedRunningTime="2026-07-24 12:49:51.652002691 +0000 UTC m=+1847.937548378" Jul 24 12:49:52.227857 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:52.226130 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:52.233595 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:52.233564 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:52.636237 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:52.636176 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:49:52.637325 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:49:52.637304 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:08.538956 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.538896 2578 patch_prober.go:28] interesting pod/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:57942->10.133.0.57:15021: read: connection reset by peer" start-of-body= Jul 24 12:50:08.539429 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.538970 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": read tcp 10.133.0.2:57942->10.133.0.57:15021: read: connection reset by peer" Jul 24 12:50:08.720036 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.719952 2578 generic.go:356] "Generic (PLEG): container finished" podID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerID="d4930d15470c0f2bd684df6f3b7f7af57da691edc2db555fc4c7beda33193761" exitCode=0 Jul 24 12:50:08.720036 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.720022 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerDied","Data":"d4930d15470c0f2bd684df6f3b7f7af57da691edc2db555fc4c7beda33193761"} Jul 24 12:50:08.720552 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.720527 2578 scope.go:117] "RemoveContainer" containerID="d4930d15470c0f2bd684df6f3b7f7af57da691edc2db555fc4c7beda33193761" Jul 24 12:50:08.721582 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.721550 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:08.721639 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.721624 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:08.721683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:08.721655 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:09.729060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:09.729026 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerStarted","Data":"91d0f808b092f424e76f9c4e7b0525549ffa3364d2c4546ebfcfd889a4ab987e"} Jul 24 12:50:10.226029 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:10.225981 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:10.226296 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:10.226272 2578 patch_prober.go:28] interesting pod/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 12:50:10.226393 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:10.226346 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 12:50:11.226600 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:11.226563 2578 patch_prober.go:28] interesting pod/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 12:50:11.227014 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:11.226619 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 12:50:12.226313 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:12.226273 2578 patch_prober.go:28] interesting pod/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 12:50:12.226516 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:12.226345 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 12:50:13.226721 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:13.226676 2578 patch_prober.go:28] interesting pod/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" start-of-body= Jul 24 12:50:13.227125 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:13.226740 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.57:15021/healthz/ready\": dial tcp 10.133.0.57:15021: connect: connection refused" Jul 24 12:50:14.229605 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:14.229569 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:14.230030 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:14.229900 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:14.230497 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:14.230476 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:29.642486 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:29.642388 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:50:29.642948 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:29.642720 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" containerID="cri-o://91d0f808b092f424e76f9c4e7b0525549ffa3364d2c4546ebfcfd889a4ab987e" gracePeriod=30 Jul 24 12:50:34.863270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.863179 2578 generic.go:356] "Generic (PLEG): container finished" podID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerID="91d0f808b092f424e76f9c4e7b0525549ffa3364d2c4546ebfcfd889a4ab987e" exitCode=0 Jul 24 12:50:34.863712 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.863236 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerDied","Data":"91d0f808b092f424e76f9c4e7b0525549ffa3364d2c4546ebfcfd889a4ab987e"} Jul 24 12:50:34.863712 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.863383 2578 scope.go:117] "RemoveContainer" containerID="d4930d15470c0f2bd684df6f3b7f7af57da691edc2db555fc4c7beda33193761" Jul 24 12:50:34.893747 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.893719 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:34.942974 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.942837 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.942994 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ddcsv\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943051 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943086 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943117 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943109 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943132 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:50:34.943290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943145 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943180 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943242 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943270 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket\") pod \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\" (UID: \"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223\") " Jul 24 12:50:34.943536 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943504 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:50:34.943628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943610 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:34.943697 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943633 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:34.943752 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.943710 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:50:34.944041 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.944013 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data" (OuterVolumeSpecName: "istio-data") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:50:34.944170 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.944024 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:50:34.945551 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.945521 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:50:34.945708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.945690 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv" (OuterVolumeSpecName: "kube-api-access-ddcsv") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "kube-api-access-ddcsv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:50:34.945879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.945857 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:50:34.945921 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:34.945868 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token" (OuterVolumeSpecName: "istio-token") pod "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" (UID: "44a920c8-1a4b-4e2b-9a1d-0b20a9c53223"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:50:35.044737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044697 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.044737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044727 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.044737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044735 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.044737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044745 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.045007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044753 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.045007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044761 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.045007 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.044771 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ddcsv\" (UniqueName: \"kubernetes.io/projected/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223-kube-api-access-ddcsv\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:50:35.870090 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.870064 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" Jul 24 12:50:35.870525 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.870059 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69" event={"ID":"44a920c8-1a4b-4e2b-9a1d-0b20a9c53223","Type":"ContainerDied","Data":"19424e1f4c9a60409ecde0ef9b2e8a3917e48109ef8b127f3f43255fddb643bf"} Jul 24 12:50:35.870525 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.870217 2578 scope.go:117] "RemoveContainer" containerID="91d0f808b092f424e76f9c4e7b0525549ffa3364d2c4546ebfcfd889a4ab987e" Jul 24 12:50:35.892531 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.892503 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:50:35.898303 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:35.898264 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e38b46e9-openshift-default-68c749bdc6-dmr69"] Jul 24 12:50:36.149582 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:36.149498 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" path="/var/lib/kubelet/pods/44a920c8-1a4b-4e2b-9a1d-0b20a9c53223/volumes" Jul 24 12:50:49.617034 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.616994 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617368 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617381 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617392 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617398 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617471 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617479 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="aee7a03c-345a-49ea-85ce-fa170579e04c" containerName="istio-proxy" Jul 24 12:50:49.617638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.617486 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:50:49.620179 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.620163 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.622251 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.622229 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f55af331-openshift-default-dockercfg-7vdzv\"" Jul 24 12:50:49.631011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.630985 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:50:49.670272 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670236 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670306 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670325 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670344 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dzpg\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670367 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670393 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670462 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670768 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670498 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.670768 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.670547 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772063 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772022 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772063 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772066 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772263 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dzpg\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772339 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772488 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772375 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772488 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772488 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772454 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772641 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772496 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772641 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772498 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772641 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772526 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772847 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772720 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772847 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772765 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.772922 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.772843 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.773140 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.773116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.774979 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.774959 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.775236 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.775218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.779654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.779631 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.779872 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.779853 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dzpg\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg\") pod \"e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:49.932956 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:49.932860 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:50.071395 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.071369 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:50:50.073133 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:50:50.073104 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a5c36fb_a399_4fe7_83ad_e2d885dd9993.slice/crio-2b8cdceb02981fbbddbf0b839997bba03f8cf922d9d476df54bb7722a281f660 WatchSource:0}: Error finding container 2b8cdceb02981fbbddbf0b839997bba03f8cf922d9d476df54bb7722a281f660: Status 404 returned error can't find the container with id 2b8cdceb02981fbbddbf0b839997bba03f8cf922d9d476df54bb7722a281f660 Jul 24 12:50:50.075240 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.075187 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:50.075324 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.075272 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:50.075324 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.075304 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:50:50.962431 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.962397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerStarted","Data":"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30"} Jul 24 12:50:50.962816 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.962471 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerStarted","Data":"2b8cdceb02981fbbddbf0b839997bba03f8cf922d9d476df54bb7722a281f660"} Jul 24 12:50:50.983035 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:50.982980 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podStartSLOduration=1.982965147 podStartE2EDuration="1.982965147s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:50:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:50:50.980817759 +0000 UTC m=+1907.266363445" watchObservedRunningTime="2026-07-24 12:50:50.982965147 +0000 UTC m=+1907.268510833" Jul 24 12:50:51.933231 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:51.933174 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:51.938284 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:51.938260 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:51.967328 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:51.967290 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:50:51.968112 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:50:51.968091 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:51:02.631611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.631562 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:51:02.632417 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.632390 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:51:02.632526 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.632422 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="44a920c8-1a4b-4e2b-9a1d-0b20a9c53223" containerName="istio-proxy" Jul 24 12:51:02.636148 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.635545 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.639301 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.639273 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f55af331\"/\"default-dockercfg-lgcr6\"" Jul 24 12:51:02.639693 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.639276 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f55af331\"/\"openshift-service-ca.crt\"" Jul 24 12:51:02.640039 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.639878 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-mt-f55af331\"/\"kube-root-ca.crt\"" Jul 24 12:51:02.640273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.640249 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-mt-f55af331\"/\"e2e-lifecycle-model-f55af331-kserve-self-signed-certs\"" Jul 24 12:51:02.642733 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.642705 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:51:02.795862 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.795812 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.796056 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.795882 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.796056 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.795911 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.796056 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.795955 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.796056 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.796037 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wrdjj\" (UniqueName: \"kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.796270 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.796081 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897148 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897052 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897148 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897098 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897158 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897227 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wrdjj\" (UniqueName: \"kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897272 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897370 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897367 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897539 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897563 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.897688 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.897674 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.902233 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.900255 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.902233 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.900773 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.909224 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.908030 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wrdjj\" (UniqueName: \"kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj\") pod \"e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:02.948854 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:02.948819 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:03.124819 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:03.124779 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:51:03.128858 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:51:03.128813 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4ff79f1b_89fd_4f08_800f_8794143896ae.slice/crio-d0e085dc8def38ee808c1c5c5cee067add9b68e702526fd36c3e22d4e137b711 WatchSource:0}: Error finding container d0e085dc8def38ee808c1c5c5cee067add9b68e702526fd36c3e22d4e137b711: Status 404 returned error can't find the container with id d0e085dc8def38ee808c1c5c5cee067add9b68e702526fd36c3e22d4e137b711 Jul 24 12:51:04.044353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:04.044321 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" event={"ID":"4ff79f1b-89fd-4f08-800f-8794143896ae","Type":"ContainerStarted","Data":"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79"} Jul 24 12:51:04.044721 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:04.044361 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" event={"ID":"4ff79f1b-89fd-4f08-800f-8794143896ae","Type":"ContainerStarted","Data":"d0e085dc8def38ee808c1c5c5cee067add9b68e702526fd36c3e22d4e137b711"} Jul 24 12:51:04.060448 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:04.060399 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" podStartSLOduration=2.060383068 podStartE2EDuration="2.060383068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:51:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:51:04.059085116 +0000 UTC m=+1920.344630802" watchObservedRunningTime="2026-07-24 12:51:04.060383068 +0000 UTC m=+1920.345928755" Jul 24 12:51:06.968750 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:06.968710 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 12:51:06.969278 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:06.968776 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 12:51:07.062765 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.062732 2578 generic.go:356] "Generic (PLEG): container finished" podID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerID="1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30" exitCode=0 Jul 24 12:51:07.062951 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.062815 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerDied","Data":"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30"} Jul 24 12:51:07.063269 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.063255 2578 scope.go:117] "RemoveContainer" containerID="1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30" Jul 24 12:51:07.064256 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.064225 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:51:07.064367 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.064309 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:51:07.064367 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:07.064358 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:51:08.070558 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:08.070517 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerStarted","Data":"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352"} Jul 24 12:51:08.933902 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:08.933866 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:51:08.934078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:08.934059 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 12:51:08.934131 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:08.934106 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 12:51:09.934318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:09.934273 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 12:51:09.934748 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:09.934336 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 12:51:10.934186 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:10.934148 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 12:51:10.934402 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:10.934228 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 12:51:11.933708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:11.933658 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" start-of-body= Jul 24 12:51:11.933894 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:11.933740 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.58:15021/healthz/ready\": dial tcp 10.133.0.58:15021: connect: connection refused" Jul 24 12:51:12.938372 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.938340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:51:12.938854 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.938749 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:51:12.939644 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.939622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:51:12.949846 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.949825 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:12.949956 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.949861 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:12.962621 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:12.962597 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:13.109284 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:13.109248 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:51:55.426315 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:51:55.426237 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:52:10.445598 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.445504 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" podUID="4ff79f1b-89fd-4f08-800f-8794143896ae" containerName="main" containerID="cri-o://630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79" gracePeriod=45 Jul 24 12:52:10.701938 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.701875 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:52:10.719927 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.719895 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wrdjj\" (UniqueName: \"kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720010 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720092 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720059 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720112 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720148 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720245 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720173 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm\") pod \"4ff79f1b-89fd-4f08-800f-8794143896ae\" (UID: \"4ff79f1b-89fd-4f08-800f-8794143896ae\") " Jul 24 12:52:10.720746 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720697 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:10.720746 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720711 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home" (OuterVolumeSpecName: "home") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:10.720746 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.720682 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache" (OuterVolumeSpecName: "model-cache") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:10.722903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.722878 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj" (OuterVolumeSpecName: "kube-api-access-wrdjj") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "kube-api-access-wrdjj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:52:10.723078 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.723061 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm" (OuterVolumeSpecName: "dshm") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:10.723269 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.723243 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "4ff79f1b-89fd-4f08-800f-8794143896ae" (UID: "4ff79f1b-89fd-4f08-800f-8794143896ae"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 12:52:10.821596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821551 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-tmp-dir\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:10.821596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821588 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-model-cache\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:10.821596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821600 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-home\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:10.821835 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821611 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4ff79f1b-89fd-4f08-800f-8794143896ae-tls-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:10.821835 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821622 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4ff79f1b-89fd-4f08-800f-8794143896ae-dshm\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:10.821835 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:10.821635 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wrdjj\" (UniqueName: \"kubernetes.io/projected/4ff79f1b-89fd-4f08-800f-8794143896ae-kube-api-access-wrdjj\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:11.412140 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.412102 2578 generic.go:356] "Generic (PLEG): container finished" podID="4ff79f1b-89fd-4f08-800f-8794143896ae" containerID="630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79" exitCode=0 Jul 24 12:52:11.412334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.412143 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" Jul 24 12:52:11.412334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.412183 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" event={"ID":"4ff79f1b-89fd-4f08-800f-8794143896ae","Type":"ContainerDied","Data":"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79"} Jul 24 12:52:11.412334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.412242 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n" event={"ID":"4ff79f1b-89fd-4f08-800f-8794143896ae","Type":"ContainerDied","Data":"d0e085dc8def38ee808c1c5c5cee067add9b68e702526fd36c3e22d4e137b711"} Jul 24 12:52:11.412334 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.412262 2578 scope.go:117] "RemoveContainer" containerID="630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79" Jul 24 12:52:11.421719 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.421699 2578 scope.go:117] "RemoveContainer" containerID="630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79" Jul 24 12:52:11.421973 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:52:11.421957 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79\": container with ID starting with 630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79 not found: ID does not exist" containerID="630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79" Jul 24 12:52:11.422017 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.421982 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79"} err="failed to get container status \"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79\": rpc error: code = NotFound desc = could not find container \"630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79\": container with ID starting with 630c56832e374fbef79499a970c4119ea0785a5cc9f03f47acc681a98ab3ba79 not found: ID does not exist" Jul 24 12:52:11.433575 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.433551 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:52:11.437662 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:11.437643 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-mt-f55af331/e2e-lifecycle-model-f55af331-kserve-66c5464787-c454n"] Jul 24 12:52:12.148579 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:12.148545 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4ff79f1b-89fd-4f08-800f-8794143896ae" path="/var/lib/kubelet/pods/4ff79f1b-89fd-4f08-800f-8794143896ae/volumes" Jul 24 12:52:17.242566 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:17.242533 2578 scope.go:117] "RemoveContainer" containerID="3c08da9b9c7529e38533667cf18b20e3f445c3c258e36d09526875bf7eb43c70" Jul 24 12:52:38.765220 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:38.765168 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:52:38.765792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:38.765448 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" containerID="cri-o://59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352" gracePeriod=30 Jul 24 12:52:42.941401 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:42.941361 2578 patch_prober.go:28] interesting pod/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 12:52:42.941792 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:42.941435 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 12:52:44.032489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.032422 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:52:44.122408 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122363 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122460 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122476 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122496 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122515 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122596 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122576 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122918 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122626 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122918 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122692 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7dzpg\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg\") pod \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\" (UID: \"5a5c36fb-a399-4fe7-83ad-e2d885dd9993\") " Jul 24 12:52:44.122918 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122899 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:44.123076 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.122990 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.123349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.123240 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:44.123349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.123310 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:52:44.123673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.123507 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data" (OuterVolumeSpecName: "istio-data") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:44.123673 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.123523 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:44.125096 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.125064 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg" (OuterVolumeSpecName: "kube-api-access-7dzpg") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "kube-api-access-7dzpg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:52:44.125279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.125259 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token" (OuterVolumeSpecName: "istio-token") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:52:44.125463 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.125425 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:52:44.125591 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.125568 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5a5c36fb-a399-4fe7-83ad-e2d885dd9993" (UID: "5a5c36fb-a399-4fe7-83ad-e2d885dd9993"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:52:44.224042 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224005 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224042 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224042 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7dzpg\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-kube-api-access-7dzpg\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224057 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224069 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224079 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224092 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224103 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.224310 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.224113 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5a5c36fb-a399-4fe7-83ad-e2d885dd9993-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:52:44.586628 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.586587 2578 generic.go:356] "Generic (PLEG): container finished" podID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerID="59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352" exitCode=0 Jul 24 12:52:44.586803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.586640 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" Jul 24 12:52:44.586803 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.586638 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerDied","Data":"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352"} Jul 24 12:52:44.586874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.586794 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr" event={"ID":"5a5c36fb-a399-4fe7-83ad-e2d885dd9993","Type":"ContainerDied","Data":"2b8cdceb02981fbbddbf0b839997bba03f8cf922d9d476df54bb7722a281f660"} Jul 24 12:52:44.586874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.586836 2578 scope.go:117] "RemoveContainer" containerID="59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352" Jul 24 12:52:44.595846 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.595825 2578 scope.go:117] "RemoveContainer" containerID="1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30" Jul 24 12:52:44.603576 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.603550 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:52:44.604927 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.604907 2578 scope.go:117] "RemoveContainer" containerID="59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352" Jul 24 12:52:44.605251 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:52:44.605230 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352\": container with ID starting with 59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352 not found: ID does not exist" containerID="59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352" Jul 24 12:52:44.605311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.605264 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352"} err="failed to get container status \"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352\": rpc error: code = NotFound desc = could not find container \"59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352\": container with ID starting with 59c5f6252960e97853173bb5397c64120f9f6501afd1d3ba97bdfb3daac7c352 not found: ID does not exist" Jul 24 12:52:44.605311 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.605290 2578 scope.go:117] "RemoveContainer" containerID="1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30" Jul 24 12:52:44.605592 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:52:44.605561 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30\": container with ID starting with 1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30 not found: ID does not exist" containerID="1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30" Jul 24 12:52:44.605694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.605595 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30"} err="failed to get container status \"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30\": rpc error: code = NotFound desc = could not find container \"1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30\": container with ID starting with 1fe6644b566065057e4b482bbd57cec38ae33527469c9ed6ae01bb935053fa30 not found: ID does not exist" Jul 24 12:52:44.609011 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:44.608989 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f55af331-openshift-default-7b9bbccb6d-rghvr"] Jul 24 12:52:46.148787 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:52:46.148751 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" path="/var/lib/kubelet/pods/5a5c36fb-a399-4fe7-83ad-e2d885dd9993/volumes" Jul 24 12:53:35.179929 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.179848 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:53:35.180475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180435 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.180475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180457 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.180475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180468 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.180475 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180476 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.180694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180487 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4ff79f1b-89fd-4f08-800f-8794143896ae" containerName="main" Jul 24 12:53:35.180694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180495 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4ff79f1b-89fd-4f08-800f-8794143896ae" containerName="main" Jul 24 12:53:35.180694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180585 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.180694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180596 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4ff79f1b-89fd-4f08-800f-8794143896ae" containerName="main" Jul 24 12:53:35.180694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.180606 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a5c36fb-a399-4fe7-83ad-e2d885dd9993" containerName="istio-proxy" Jul 24 12:53:35.184919 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.184896 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.187318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.187297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-dd4baf-openshift-default-dockercfg-x45td\"" Jul 24 12:53:35.198095 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.198070 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:53:35.275387 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275353 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275580 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275396 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275580 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275420 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275580 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275580 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275532 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275580 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275563 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275627 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275707 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-29lf5\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.275829 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.275752 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376256 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376224 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376591 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-29lf5\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376623 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376647 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376669 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376700 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376683 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376707 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376738 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376769 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.376903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.376802 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.377148 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.377123 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.377330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.377260 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.377330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.377283 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.377601 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.377580 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.379203 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.379169 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.379276 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.379208 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.384279 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.384258 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.384633 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.384610 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-29lf5\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5\") pod \"e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.495132 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.495042 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:35.639371 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.639343 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:53:35.641472 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:53:35.641441 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84cb73b9_80b0_480d_9bff_8480e6c7f8c7.slice/crio-1f170a5e4695e7545f7de7895afa35262227506b630d156c5fdf54fce5c37e1b WatchSource:0}: Error finding container 1f170a5e4695e7545f7de7895afa35262227506b630d156c5fdf54fce5c37e1b: Status 404 returned error can't find the container with id 1f170a5e4695e7545f7de7895afa35262227506b630d156c5fdf54fce5c37e1b Jul 24 12:53:35.643696 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.643660 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:35.643758 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.643733 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:35.643796 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.643764 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:35.848622 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.848584 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerStarted","Data":"396f7c8953c7d6499d1a75a3756949866b8c7a03810bf43d839dcc3c362802ff"} Jul 24 12:53:35.848806 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.848652 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerStarted","Data":"1f170a5e4695e7545f7de7895afa35262227506b630d156c5fdf54fce5c37e1b"} Jul 24 12:53:35.872118 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:35.872056 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podStartSLOduration=0.872034664 podStartE2EDuration="872.034664ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:53:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:53:35.868639868 +0000 UTC m=+2072.154185576" watchObservedRunningTime="2026-07-24 12:53:35.872034664 +0000 UTC m=+2072.157580349" Jul 24 12:53:36.495775 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:36.495736 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:36.497357 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:36.497329 2578 patch_prober.go:28] interesting pod/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 12:53:36.497480 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:36.497377 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 12:53:37.500269 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:37.500237 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:37.858539 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:37.858511 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:37.859627 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:37.859606 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:43.057493 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.057460 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:53:43.060978 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.060956 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.063663 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.063641 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-fc0bc9-openshift-default-dockercfg-zx6sb\"" Jul 24 12:53:43.082542 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.082512 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:53:43.150204 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150166 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150215 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150286 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150327 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150347 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150373 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150415 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150444 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqkmr\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.150554 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.150474 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.250943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.250907 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.250943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.250948 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.250967 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lqkmr\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251002 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251069 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251095 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251135 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251170 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251531 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251377 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251531 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251405 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251531 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251495 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251547 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.251654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251590 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.252003 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.251953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.253569 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.253548 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.253692 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.253672 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.262300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.262270 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqkmr\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.262431 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.262352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token\") pod \"e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.371873 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.371783 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:43.528789 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.528764 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:53:43.531547 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:53:43.531519 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc0489c54_1807_4306_9e0a_568fcbcdd15f.slice/crio-f6aee66fecd977434a712490d78a9701498f91e34f36ea5b69f91ab993c054e6 WatchSource:0}: Error finding container f6aee66fecd977434a712490d78a9701498f91e34f36ea5b69f91ab993c054e6: Status 404 returned error can't find the container with id f6aee66fecd977434a712490d78a9701498f91e34f36ea5b69f91ab993c054e6 Jul 24 12:53:43.533690 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.533656 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:43.533769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.533720 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:43.533769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.533749 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:43.892349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.892315 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerStarted","Data":"e19f37dd8c734ebfdd729817a86fbc3b4783e9ab5a9180fc14821fad50ad373c"} Jul 24 12:53:43.892533 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.892369 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerStarted","Data":"f6aee66fecd977434a712490d78a9701498f91e34f36ea5b69f91ab993c054e6"} Jul 24 12:53:43.913181 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:43.913116 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podStartSLOduration=0.913095677 podStartE2EDuration="913.095677ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:53:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:53:43.912783222 +0000 UTC m=+2080.198328919" watchObservedRunningTime="2026-07-24 12:53:43.913095677 +0000 UTC m=+2080.198641363" Jul 24 12:53:44.372992 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:44.372948 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:44.377978 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:44.377953 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:44.917443 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:44.917402 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:44.927015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:44.926968 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:51.959594 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.959559 2578 generic.go:356] "Generic (PLEG): container finished" podID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerID="396f7c8953c7d6499d1a75a3756949866b8c7a03810bf43d839dcc3c362802ff" exitCode=0 Jul 24 12:53:51.960013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.959601 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerDied","Data":"396f7c8953c7d6499d1a75a3756949866b8c7a03810bf43d839dcc3c362802ff"} Jul 24 12:53:51.960013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.959984 2578 scope.go:117] "RemoveContainer" containerID="396f7c8953c7d6499d1a75a3756949866b8c7a03810bf43d839dcc3c362802ff" Jul 24 12:53:51.961090 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.961061 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:51.961148 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.961128 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:51.961187 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:51.961158 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:52.859359 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:52.859325 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:52.967424 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:52.967390 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerStarted","Data":"7bff7e7e020dfb3f0060bd254b098846fdbf1f4924fd6257f067c29742a23735"} Jul 24 12:53:53.496253 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:53.496217 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:53.497465 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:53.497437 2578 patch_prober.go:28] interesting pod/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 12:53:53.497595 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:53.497486 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 12:53:54.495760 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:54.495719 2578 patch_prober.go:28] interesting pod/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 12:53:54.496150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:54.495778 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 12:53:55.495865 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:55.495825 2578 patch_prober.go:28] interesting pod/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 12:53:55.496364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:55.495900 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 12:53:56.496470 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:56.496425 2578 patch_prober.go:28] interesting pod/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body= Jul 24 12:53:56.496966 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:56.496490 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" Jul 24 12:53:57.500579 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.500550 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:57.501069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.501009 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:57.501716 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.501694 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:53:57.995689 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.995650 2578 generic.go:356] "Generic (PLEG): container finished" podID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerID="e19f37dd8c734ebfdd729817a86fbc3b4783e9ab5a9180fc14821fad50ad373c" exitCode=0 Jul 24 12:53:57.995874 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.995724 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerDied","Data":"e19f37dd8c734ebfdd729817a86fbc3b4783e9ab5a9180fc14821fad50ad373c"} Jul 24 12:53:57.996345 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.996328 2578 scope.go:117] "RemoveContainer" containerID="e19f37dd8c734ebfdd729817a86fbc3b4783e9ab5a9180fc14821fad50ad373c" Jul 24 12:53:57.997400 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.997372 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:57.997504 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.997450 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:57.997504 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:57.997496 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:53:59.005865 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:59.005826 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerStarted","Data":"f4758de3aa40b48e29dcd2c448e296793c80a654aff0f1fbdd3f70b8d6c53657"} Jul 24 12:53:59.372611 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:59.372575 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:53:59.373719 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:59.373695 2578 patch_prober.go:28] interesting pod/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 12:53:59.373780 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:59.373750 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 12:53:59.917889 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:53:59.917808 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:54:00.372397 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:00.372356 2578 patch_prober.go:28] interesting pod/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 12:54:00.372784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:00.372416 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 12:54:01.372836 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:01.372793 2578 patch_prober.go:28] interesting pod/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 12:54:01.373255 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:01.372857 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 12:54:02.372531 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:02.372494 2578 patch_prober.go:28] interesting pod/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" start-of-body= Jul 24 12:54:02.372711 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:02.372557 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.61:15021/healthz/ready\": dial tcp 10.133.0.61:15021: connect: connection refused" Jul 24 12:54:03.376353 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:03.376323 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:54:03.377447 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:54:03.377426 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:55:21.425522 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.425481 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4"] Jul 24 12:55:21.428939 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.428922 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:21.430963 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.430941 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 24 12:55:21.431090 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.430941 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-zw8hg\"" Jul 24 12:55:21.439494 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.439462 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4"] Jul 24 12:55:21.542307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.542262 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4\" (UID: \"84e3e81e-86d8-4230-930d-a750a2e4f8ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:21.643741 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.643701 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4\" (UID: \"84e3e81e-86d8-4230-930d-a750a2e4f8ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:21.644446 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.644421 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4\" (UID: \"84e3e81e-86d8-4230-930d-a750a2e4f8ef\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:21.748997 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.748903 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:21.879602 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:21.879564 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4"] Jul 24 12:55:21.882737 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:55:21.882705 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84e3e81e_86d8_4230_930d_a750a2e4f8ef.slice/crio-589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc WatchSource:0}: Error finding container 589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc: Status 404 returned error can't find the container with id 589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc Jul 24 12:55:22.456086 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:22.455979 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" event={"ID":"84e3e81e-86d8-4230-930d-a750a2e4f8ef","Type":"ContainerStarted","Data":"abbfead1b207be0f97c4fe8830ac268859568dddfdaa24008f2e87ef8802d47d"} Jul 24 12:55:22.456086 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:22.456043 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" event={"ID":"84e3e81e-86d8-4230-930d-a750a2e4f8ef","Type":"ContainerStarted","Data":"589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc"} Jul 24 12:55:22.471716 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:22.471664 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" podStartSLOduration=1.471649692 podStartE2EDuration="1.471649692s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:55:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:55:22.469622284 +0000 UTC m=+2178.755167969" watchObservedRunningTime="2026-07-24 12:55:22.471649692 +0000 UTC m=+2178.757195376" Jul 24 12:55:23.463055 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:23.463022 2578 generic.go:356] "Generic (PLEG): container finished" podID="84e3e81e-86d8-4230-930d-a750a2e4f8ef" containerID="abbfead1b207be0f97c4fe8830ac268859568dddfdaa24008f2e87ef8802d47d" exitCode=0 Jul 24 12:55:23.463471 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:23.463115 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" event={"ID":"84e3e81e-86d8-4230-930d-a750a2e4f8ef","Type":"ContainerDied","Data":"abbfead1b207be0f97c4fe8830ac268859568dddfdaa24008f2e87ef8802d47d"} Jul 24 12:55:24.608477 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:24.608453 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:24.670806 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:24.670769 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca\") pod \"84e3e81e-86d8-4230-930d-a750a2e4f8ef\" (UID: \"84e3e81e-86d8-4230-930d-a750a2e4f8ef\") " Jul 24 12:55:24.671214 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:24.671170 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "84e3e81e-86d8-4230-930d-a750a2e4f8ef" (UID: "84e3e81e-86d8-4230-930d-a750a2e4f8ef"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:55:24.772269 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:24.772232 2578 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/84e3e81e-86d8-4230-930d-a750a2e4f8ef-maas-api-service-ca\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:25.475394 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:25.475358 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" Jul 24 12:55:25.475561 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:25.475357 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4" event={"ID":"84e3e81e-86d8-4230-930d-a750a2e4f8ef","Type":"ContainerDied","Data":"589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc"} Jul 24 12:55:25.475561 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:25.475480 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="589db47f4d4c9ef3119e560d914418abf3c8fa1ee13f37cb27a51bd303eebabc" Jul 24 12:55:36.571470 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:36.571432 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4"] Jul 24 12:55:36.574989 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:36.574962 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-dd4baf-9srh4"] Jul 24 12:55:38.148938 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:38.148908 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84e3e81e-86d8-4230-930d-a750a2e4f8ef" path="/var/lib/kubelet/pods/84e3e81e-86d8-4230-930d-a750a2e4f8ef/volumes" Jul 24 12:55:43.427323 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:43.427290 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:55:43.427822 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:43.427533 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" containerID="cri-o://7bff7e7e020dfb3f0060bd254b098846fdbf1f4924fd6257f067c29742a23735" gracePeriod=30 Jul 24 12:55:48.610870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.610831 2578 generic.go:356] "Generic (PLEG): container finished" podID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerID="7bff7e7e020dfb3f0060bd254b098846fdbf1f4924fd6257f067c29742a23735" exitCode=0 Jul 24 12:55:48.611348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.610897 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerDied","Data":"7bff7e7e020dfb3f0060bd254b098846fdbf1f4924fd6257f067c29742a23735"} Jul 24 12:55:48.611348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.610962 2578 scope.go:117] "RemoveContainer" containerID="396f7c8953c7d6499d1a75a3756949866b8c7a03810bf43d839dcc3c362802ff" Jul 24 12:55:48.684352 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.684322 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:55:48.799704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799673 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.799887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799735 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.799887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799779 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.799887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799803 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.799887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799818 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.799887 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799837 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.800146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799993 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.800146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800037 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-29lf5\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.800146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800071 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data\") pod \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\" (UID: \"84cb73b9-80b0-480d-9bff-8480e6c7f8c7\") " Jul 24 12:55:48.800146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.799995 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:55:48.800146 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800087 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:55:48.800420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800145 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:55:48.800420 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800298 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:55:48.800527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800461 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.800527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800479 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.800527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800492 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.800527 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800505 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.800882 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.800791 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data" (OuterVolumeSpecName: "istio-data") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:55:48.802183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.802160 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:55:48.802183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.802164 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token" (OuterVolumeSpecName: "istio-token") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:55:48.802646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.802619 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5" (OuterVolumeSpecName: "kube-api-access-29lf5") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "kube-api-access-29lf5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:55:48.802646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.802613 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "84cb73b9-80b0-480d-9bff-8480e6c7f8c7" (UID: "84cb73b9-80b0-480d-9bff-8480e6c7f8c7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:55:48.901659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.901620 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.901659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.901650 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.901659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.901659 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.901659 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.901668 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-29lf5\" (UniqueName: \"kubernetes.io/projected/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-kube-api-access-29lf5\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:48.901921 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:48.901677 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/84cb73b9-80b0-480d-9bff-8480e6c7f8c7-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:55:49.617343 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:49.617298 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" event={"ID":"84cb73b9-80b0-480d-9bff-8480e6c7f8c7","Type":"ContainerDied","Data":"1f170a5e4695e7545f7de7895afa35262227506b630d156c5fdf54fce5c37e1b"} Jul 24 12:55:49.617815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:49.617353 2578 scope.go:117] "RemoveContainer" containerID="7bff7e7e020dfb3f0060bd254b098846fdbf1f4924fd6257f067c29742a23735" Jul 24 12:55:49.617815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:49.617315 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk" Jul 24 12:55:49.640149 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:49.640115 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:55:49.643561 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:49.643535 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-dd4baf-openshift-default-5bcf886748-vcbpk"] Jul 24 12:55:50.148318 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:55:50.148289 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" path="/var/lib/kubelet/pods/84cb73b9-80b0-480d-9bff-8480e6c7f8c7/volumes" Jul 24 12:56:05.500348 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:05.500311 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:56:05.500845 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:05.500557 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" containerID="cri-o://f4758de3aa40b48e29dcd2c448e296793c80a654aff0f1fbdd3f70b8d6c53657" gracePeriod=30 Jul 24 12:56:06.440080 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440047 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:56:06.440425 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440410 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:06.440489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440427 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:06.440489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440440 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84e3e81e-86d8-4230-930d-a750a2e4f8ef" containerName="revoke-keys" Jul 24 12:56:06.440489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440445 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84e3e81e-86d8-4230-930d-a750a2e4f8ef" containerName="revoke-keys" Jul 24 12:56:06.440650 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440499 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:06.440650 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.440511 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84e3e81e-86d8-4230-930d-a750a2e4f8ef" containerName="revoke-keys" Jul 24 12:56:06.444784 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.444765 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.447093 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.447072 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-f394b5-openshift-default-dockercfg-sx78f\"" Jul 24 12:56:06.456432 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.456407 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:56:06.562023 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.561987 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562023 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562023 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562061 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562127 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562164 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562214 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562237 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562314 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.562510 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.562330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h6wqq\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663378 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663342 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663378 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663383 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663401 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663467 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663483 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h6wqq\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663508 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663534 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663591 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663632 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663619 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663941 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663813 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.663997 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.663961 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.664051 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.664003 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.664160 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.664137 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.664359 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.664301 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.665938 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.665909 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.666172 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.666153 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.678244 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.678217 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.678512 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.678493 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h6wqq\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq\") pod \"e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.755735 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.755639 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:06.901172 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.901144 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:56:06.902948 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:56:06.902924 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a55e666_b46c_430e_a1e6_edfa76f56634.slice/crio-404c98f9e326c4247c9c0d6e0314b78c812c3f05651b503c314c7b6209630280 WatchSource:0}: Error finding container 404c98f9e326c4247c9c0d6e0314b78c812c3f05651b503c314c7b6209630280: Status 404 returned error can't find the container with id 404c98f9e326c4247c9c0d6e0314b78c812c3f05651b503c314c7b6209630280 Jul 24 12:56:06.904981 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.904948 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:06.905069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.905025 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:06.905133 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:06.905069 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:07.714556 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:07.714511 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerStarted","Data":"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a"} Jul 24 12:56:07.714946 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:07.714577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerStarted","Data":"404c98f9e326c4247c9c0d6e0314b78c812c3f05651b503c314c7b6209630280"} Jul 24 12:56:07.737556 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:07.737501 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podStartSLOduration=1.737485865 podStartE2EDuration="1.737485865s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:56:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:56:07.735141835 +0000 UTC m=+2224.020687521" watchObservedRunningTime="2026-07-24 12:56:07.737485865 +0000 UTC m=+2224.023031548" Jul 24 12:56:07.755769 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:07.755734 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:07.760645 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:07.760622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:08.720519 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:08.720483 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:08.721440 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:08.721418 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:10.733239 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.733183 2578 generic.go:356] "Generic (PLEG): container finished" podID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerID="f4758de3aa40b48e29dcd2c448e296793c80a654aff0f1fbdd3f70b8d6c53657" exitCode=0 Jul 24 12:56:10.733239 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.733235 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerDied","Data":"f4758de3aa40b48e29dcd2c448e296793c80a654aff0f1fbdd3f70b8d6c53657"} Jul 24 12:56:10.733683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.733300 2578 scope.go:117] "RemoveContainer" containerID="e19f37dd8c734ebfdd729817a86fbc3b4783e9ab5a9180fc14821fad50ad373c" Jul 24 12:56:10.767753 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.767698 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:56:10.904156 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904121 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904214 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904244 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904264 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904361 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904334 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lqkmr\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904392 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904437 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904499 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904530 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert\") pod \"c0489c54-1807-4306-9e0a-568fcbcdd15f\" (UID: \"c0489c54-1807-4306-9e0a-568fcbcdd15f\") " Jul 24 12:56:10.904586 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904503 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:56:10.904843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904579 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:56:10.904843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904838 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:10.904947 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904855 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:10.904996 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904964 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:56:10.905055 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.904992 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data" (OuterVolumeSpecName: "istio-data") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:56:10.905266 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.905237 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:56:10.906945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.906914 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:56:10.906945 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.906934 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:56:10.907108 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.907086 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr" (OuterVolumeSpecName: "kube-api-access-lqkmr") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "kube-api-access-lqkmr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:56:10.907410 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:10.907394 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token" (OuterVolumeSpecName: "istio-token") pod "c0489c54-1807-4306-9e0a-568fcbcdd15f" (UID: "c0489c54-1807-4306-9e0a-568fcbcdd15f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:56:11.006480 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006440 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006480 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006475 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006490 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lqkmr\" (UniqueName: \"kubernetes.io/projected/c0489c54-1807-4306-9e0a-568fcbcdd15f-kube-api-access-lqkmr\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006503 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006519 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c0489c54-1807-4306-9e0a-568fcbcdd15f-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006532 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c0489c54-1807-4306-9e0a-568fcbcdd15f-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.006704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.006544 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c0489c54-1807-4306-9e0a-568fcbcdd15f-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:56:11.739782 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.739748 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" Jul 24 12:56:11.740183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.739803 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6" event={"ID":"c0489c54-1807-4306-9e0a-568fcbcdd15f","Type":"ContainerDied","Data":"f6aee66fecd977434a712490d78a9701498f91e34f36ea5b69f91ab993c054e6"} Jul 24 12:56:11.740183 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.739863 2578 scope.go:117] "RemoveContainer" containerID="f4758de3aa40b48e29dcd2c448e296793c80a654aff0f1fbdd3f70b8d6c53657" Jul 24 12:56:11.779326 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.779294 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:56:11.790849 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:11.790816 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-fc0bc9-openshift-default-65ddd98b46-4mzb6"] Jul 24 12:56:12.149363 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:12.149331 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" path="/var/lib/kubelet/pods/c0489c54-1807-4306-9e0a-568fcbcdd15f/volumes" Jul 24 12:56:15.308373 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308334 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308696 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308708 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308717 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308721 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308781 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="84cb73b9-80b0-480d-9bff-8480e6c7f8c7" containerName="istio-proxy" Jul 24 12:56:15.308839 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.308790 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:56:15.313538 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.313517 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.316529 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.316492 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-aa0820-openshift-default-dockercfg-84ts5\"" Jul 24 12:56:15.349015 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.348985 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:56:15.443364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443330 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443366 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443392 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443410 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443428 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443486 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443585 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443566 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443746 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443589 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nq25q\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.443746 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.443654 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544434 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544474 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544475 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544516 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544540 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544567 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544840 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544726 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544840 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544840 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544832 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nq25q\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544891 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544901 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.544993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.544932 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.545144 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.545087 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.545287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.545268 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.545501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.545482 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.547076 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.547053 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.547215 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.547180 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.556843 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.556821 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.560000 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.559950 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nq25q\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q\") pod \"e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.623740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.623703 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:15.781729 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.781691 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:56:15.783612 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:56:15.783576 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26d235e0_ac18_45a1_9ddf_efef47fed551.slice/crio-08da52220536922f4c15de7131fec385a596f35349ced3e723a639cec0d2d540 WatchSource:0}: Error finding container 08da52220536922f4c15de7131fec385a596f35349ced3e723a639cec0d2d540: Status 404 returned error can't find the container with id 08da52220536922f4c15de7131fec385a596f35349ced3e723a639cec0d2d540 Jul 24 12:56:15.786122 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.786079 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:15.786267 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.786155 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:15.786267 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:15.786213 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:16.767794 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:16.767759 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerStarted","Data":"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93"} Jul 24 12:56:16.768235 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:16.767812 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerStarted","Data":"08da52220536922f4c15de7131fec385a596f35349ced3e723a639cec0d2d540"} Jul 24 12:56:16.809624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:16.809529 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podStartSLOduration=1.809490763 podStartE2EDuration="1.809490763s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:56:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:56:16.805244601 +0000 UTC m=+2233.090790284" watchObservedRunningTime="2026-07-24 12:56:16.809490763 +0000 UTC m=+2233.095036450" Jul 24 12:56:17.624335 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:17.624240 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:17.629831 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:17.629804 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:17.772612 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:17.772583 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:17.773739 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:17.773720 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:23.721676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.721582 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 12:56:23.721676 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.721645 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 12:56:23.806207 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.806162 2578 generic.go:356] "Generic (PLEG): container finished" podID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerID="84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a" exitCode=0 Jul 24 12:56:23.806378 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.806218 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerDied","Data":"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a"} Jul 24 12:56:23.806612 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.806594 2578 scope.go:117] "RemoveContainer" containerID="84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a" Jul 24 12:56:23.807686 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.807655 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:23.807761 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.807724 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:23.807761 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:23.807756 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:24.822438 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:24.822322 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerStarted","Data":"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c"} Jul 24 12:56:25.756064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:25.756027 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:25.756266 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:25.756233 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 12:56:25.756355 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:25.756283 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 12:56:26.756405 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:26.756362 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 12:56:26.756879 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:26.756420 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 12:56:27.756815 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:27.756773 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 12:56:27.757237 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:27.756834 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 12:56:28.756168 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:28.756125 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body= Jul 24 12:56:28.756381 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:28.756238 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" Jul 24 12:56:29.761577 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:29.761546 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:29.761983 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:29.761853 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:29.762516 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:29.762497 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:56:36.890069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.890034 2578 generic.go:356] "Generic (PLEG): container finished" podID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerID="6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93" exitCode=0 Jul 24 12:56:36.890557 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.890076 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerDied","Data":"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93"} Jul 24 12:56:36.890557 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.890479 2578 scope.go:117] "RemoveContainer" containerID="6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93" Jul 24 12:56:36.891534 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.891502 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:36.891622 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.891584 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:36.891622 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:36.891617 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:56:37.896825 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:37.896789 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerStarted","Data":"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58"} Jul 24 12:56:38.624624 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:38.624576 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:38.624823 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:38.624801 2578 patch_prober.go:28] interesting pod/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 24 12:56:38.624870 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:38.624852 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 24 12:56:39.624634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:39.624595 2578 patch_prober.go:28] interesting pod/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 24 12:56:39.625044 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:39.624655 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 24 12:56:40.624213 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:40.624155 2578 patch_prober.go:28] interesting pod/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 24 12:56:40.624423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:40.624258 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 24 12:56:41.624300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:41.624263 2578 patch_prober.go:28] interesting pod/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" start-of-body= Jul 24 12:56:41.624705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:41.624325 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.64:15021/healthz/ready\": dial tcp 10.133.0.64:15021: connect: connection refused" Jul 24 12:56:42.628210 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:42.628164 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:42.628710 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:42.628698 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:42.629363 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:42.629340 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:56:58.951109 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:58.951077 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:56:58.951654 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:58.951313 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" containerID="cri-o://48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c" gracePeriod=30 Jul 24 12:56:59.764600 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:59.764564 2578 patch_prober.go:28] interesting pod/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 12:56:59.764813 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:56:59.764640 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 12:57:04.198502 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.198473 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:57:04.293688 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293658 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.293971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293711 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.293971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293734 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.293971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293754 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.293971 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293876 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.293997 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294027 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294065 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294088 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-h6wqq\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq\") pod \"2a55e666-b46c-430e-a1e6-edfa76f56634\" (UID: \"2a55e666-b46c-430e-a1e6-edfa76f56634\") " Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294120 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294126 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:57:04.294225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294136 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:04.294737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294287 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:04.294737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294440 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2a55e666-b46c-430e-a1e6-edfa76f56634-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.294737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294460 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.294737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294474 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.294737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294487 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.294994 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.294788 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data" (OuterVolumeSpecName: "istio-data") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:04.296162 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.296131 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token" (OuterVolumeSpecName: "istio-token") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:57:04.296287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.296175 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:04.296708 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.296690 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:57:04.296763 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.296703 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq" (OuterVolumeSpecName: "kube-api-access-h6wqq") pod "2a55e666-b46c-430e-a1e6-edfa76f56634" (UID: "2a55e666-b46c-430e-a1e6-edfa76f56634"). InnerVolumeSpecName "kube-api-access-h6wqq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:57:04.395694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.395655 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.395694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.395689 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.395694 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.395702 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-h6wqq\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-kube-api-access-h6wqq\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.395943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.395714 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:04.395943 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:04.395724 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2a55e666-b46c-430e-a1e6-edfa76f56634-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:05.047069 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.047028 2578 generic.go:356] "Generic (PLEG): container finished" podID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerID="48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c" exitCode=0 Jul 24 12:57:05.047300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.047107 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" Jul 24 12:57:05.047300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.047106 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerDied","Data":"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c"} Jul 24 12:57:05.047300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.047273 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l" event={"ID":"2a55e666-b46c-430e-a1e6-edfa76f56634","Type":"ContainerDied","Data":"404c98f9e326c4247c9c0d6e0314b78c812c3f05651b503c314c7b6209630280"} Jul 24 12:57:05.047300 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.047297 2578 scope.go:117] "RemoveContainer" containerID="48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c" Jul 24 12:57:05.057273 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.057255 2578 scope.go:117] "RemoveContainer" containerID="84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a" Jul 24 12:57:05.065653 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.065507 2578 scope.go:117] "RemoveContainer" containerID="48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c" Jul 24 12:57:05.065805 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:57:05.065786 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c\": container with ID starting with 48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c not found: ID does not exist" containerID="48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c" Jul 24 12:57:05.065848 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.065816 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c"} err="failed to get container status \"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c\": rpc error: code = NotFound desc = could not find container \"48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c\": container with ID starting with 48a243f107b59cfa573a4f057494aa1f591b35b86faf940106018d36433e947c not found: ID does not exist" Jul 24 12:57:05.065848 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.065835 2578 scope.go:117] "RemoveContainer" containerID="84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a" Jul 24 12:57:05.066090 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:57:05.066068 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a\": container with ID starting with 84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a not found: ID does not exist" containerID="84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a" Jul 24 12:57:05.066153 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.066100 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a"} err="failed to get container status \"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a\": rpc error: code = NotFound desc = could not find container \"84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a\": container with ID starting with 84a72ca5c5d6c1f321a2c0faaccb631fd86a79cbb0e119fda920f414ae92c83a not found: ID does not exist" Jul 24 12:57:05.070112 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.070086 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:57:05.074671 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:05.074645 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-f394b5-openshift-default-554fb554bb-n7v4l"] Jul 24 12:57:06.147924 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:06.147891 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" path="/var/lib/kubelet/pods/2a55e666-b46c-430e-a1e6-edfa76f56634/volumes" Jul 24 12:57:26.306570 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:26.306534 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:57:26.307026 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:26.306770 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" containerID="cri-o://9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58" gracePeriod=30 Jul 24 12:57:27.266964 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.266923 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 12:57:27.267386 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267365 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:57:27.267501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267388 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:57:27.267501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267408 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:27.267501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267416 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:27.267646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267506 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:27.267646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267519 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:27.267646 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.267531 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="c0489c54-1807-4306-9e0a-568fcbcdd15f" containerName="istio-proxy" Jul 24 12:57:27.271665 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.271644 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.274364 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.274338 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-d15f64-openshift-default-dockercfg-lx25m\"" Jul 24 12:57:27.289515 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.289482 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 12:57:27.406544 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406506 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406548 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406615 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406644 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406694 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4fqtz\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406744 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406824 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.406993 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.406892 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508095 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508049 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508111 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508139 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508176 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508231 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508330 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508255 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508634 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508610 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508696 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508305 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508737 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508724 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508788 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508728 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508834 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508796 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508834 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508800 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4fqtz\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.508898 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.508684 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.509371 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.509352 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.511111 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.511080 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.511424 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.511400 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.517228 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.517145 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.517306 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.517232 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4fqtz\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz\") pod \"e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.583008 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.582966 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:27.630838 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.630800 2578 patch_prober.go:28] interesting pod/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 12:57:27.630990 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.630867 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 12:57:27.728485 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.728450 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 12:57:27.729400 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:57:27.729374 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3619f5f2_545c_4ed0_84f9_b8c4d63bdf70.slice/crio-9f03c78e2932e07d715eef2a647a1561f15e1b28b74fd5bf70909b0891084158 WatchSource:0}: Error finding container 9f03c78e2932e07d715eef2a647a1561f15e1b28b74fd5bf70909b0891084158: Status 404 returned error can't find the container with id 9f03c78e2932e07d715eef2a647a1561f15e1b28b74fd5bf70909b0891084158 Jul 24 12:57:27.731710 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.731668 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:27.731808 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.731752 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:27.731808 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:27.731792 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:28.169019 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:28.168930 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerStarted","Data":"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec"} Jul 24 12:57:28.169019 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:28.168985 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerStarted","Data":"9f03c78e2932e07d715eef2a647a1561f15e1b28b74fd5bf70909b0891084158"} Jul 24 12:57:28.188734 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:28.188684 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podStartSLOduration=1.18866816 podStartE2EDuration="1.18866816s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:57:27 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:57:28.187269958 +0000 UTC m=+2304.472815662" watchObservedRunningTime="2026-07-24 12:57:28.18866816 +0000 UTC m=+2304.474213844" Jul 24 12:57:28.584106 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:28.584066 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:28.589013 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:28.588989 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:29.175182 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:29.175148 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:29.176169 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:29.176152 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:31.566296 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.566270 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:57:31.753906 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.753826 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.753906 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.753859 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.753906 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.753882 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nq25q\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754123 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.753971 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754123 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754044 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754123 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754076 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754123 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754100 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754343 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754134 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754343 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754138 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:31.754343 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754164 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token\") pod \"26d235e0-ac18-45a1-9ddf-efef47fed551\" (UID: \"26d235e0-ac18-45a1-9ddf-efef47fed551\") " Jul 24 12:57:31.754501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:31.754501 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754451 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 12:57:31.754607 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754562 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.754680 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754659 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:31.754742 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754582 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.754742 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.754702 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/26d235e0-ac18-45a1-9ddf-efef47fed551-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.755084 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.755058 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data" (OuterVolumeSpecName: "istio-data") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:31.756423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.756400 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 12:57:31.756423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.756401 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 12:57:31.756423 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.756419 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token" (OuterVolumeSpecName: "istio-token") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:57:31.756589 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.756431 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q" (OuterVolumeSpecName: "kube-api-access-nq25q") pod "26d235e0-ac18-45a1-9ddf-efef47fed551" (UID: "26d235e0-ac18-45a1-9ddf-efef47fed551"). InnerVolumeSpecName "kube-api-access-nq25q". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 12:57:31.855727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855692 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.855727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855719 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.855727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855729 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/26d235e0-ac18-45a1-9ddf-efef47fed551-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.855727 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855737 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.855987 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855745 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/26d235e0-ac18-45a1-9ddf-efef47fed551-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:31.855987 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:31.855754 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nq25q\" (UniqueName: \"kubernetes.io/projected/26d235e0-ac18-45a1-9ddf-efef47fed551-kube-api-access-nq25q\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 12:57:32.192150 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.192113 2578 generic.go:356] "Generic (PLEG): container finished" podID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerID="9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58" exitCode=0 Jul 24 12:57:32.192349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.192158 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerDied","Data":"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58"} Jul 24 12:57:32.192349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.192234 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" event={"ID":"26d235e0-ac18-45a1-9ddf-efef47fed551","Type":"ContainerDied","Data":"08da52220536922f4c15de7131fec385a596f35349ced3e723a639cec0d2d540"} Jul 24 12:57:32.192349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.192258 2578 scope.go:117] "RemoveContainer" containerID="9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58" Jul 24 12:57:32.192349 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.192171 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp" Jul 24 12:57:32.201666 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.201651 2578 scope.go:117] "RemoveContainer" containerID="6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93" Jul 24 12:57:32.209098 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.209072 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:57:32.210371 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.210348 2578 scope.go:117] "RemoveContainer" containerID="9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58" Jul 24 12:57:32.210661 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:57:32.210644 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58\": container with ID starting with 9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58 not found: ID does not exist" containerID="9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58" Jul 24 12:57:32.210705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.210674 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58"} err="failed to get container status \"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58\": rpc error: code = NotFound desc = could not find container \"9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58\": container with ID starting with 9d523920a5a37469eb807c1e5c8105816e3cdde7de3920217e840e297c5d3a58 not found: ID does not exist" Jul 24 12:57:32.210705 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.210692 2578 scope.go:117] "RemoveContainer" containerID="6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93" Jul 24 12:57:32.210950 ip-10-0-132-126 kubenswrapper[2578]: E0724 12:57:32.210929 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93\": container with ID starting with 6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93 not found: ID does not exist" containerID="6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93" Jul 24 12:57:32.210996 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.210956 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93"} err="failed to get container status \"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93\": rpc error: code = NotFound desc = could not find container \"6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93\": container with ID starting with 6abee0faa56fdb754208b2dd85909dfb81fe367948c7f86b7d5918821598dd93 not found: ID does not exist" Jul 24 12:57:32.215290 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:32.215270 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-aa0820-openshift-default-665ddbbc94-f45gp"] Jul 24 12:57:34.148683 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:34.148648 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" path="/var/lib/kubelet/pods/26d235e0-ac18-45a1-9ddf-efef47fed551/volumes" Jul 24 12:57:35.388667 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.388627 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 12:57:35.389127 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389108 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.389127 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389127 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.389246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389139 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.389246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389145 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.389246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389157 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:35.389246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389162 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a55e666-b46c-430e-a1e6-edfa76f56634" containerName="istio-proxy" Jul 24 12:57:35.389246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389242 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.389397 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.389254 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="26d235e0-ac18-45a1-9ddf-efef47fed551" containerName="istio-proxy" Jul 24 12:57:35.393233 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.393214 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.395377 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.395355 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-c6697a-openshift-default-dockercfg-rnlqt\"" Jul 24 12:57:35.406060 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.406032 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 12:57:35.489734 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489696 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.489933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489770 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.489933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.489933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489835 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.489933 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489888 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.490158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.489992 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.490158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.490048 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.490158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.490075 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q26t\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.490158 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.490117 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590601 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590638 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590643 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q26t\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590682 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590745 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590791 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590822 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590846 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.590895 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590877 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591188 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.590908 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591188 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.591115 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591445 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.591417 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.591456 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.591474 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.591573 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.591534 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.593903 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.593873 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.594088 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.594073 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.599213 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.599161 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.599428 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.599407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q26t\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t\") pod \"e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.703316 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.703224 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:35.846431 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.846407 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 12:57:35.848542 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:57:35.848508 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2190551a_80a6_4814_b040_e5a07aab6cf0.slice/crio-73b90fe0f953c0e004e27aed5656dd4581efdd66a20c0a5db52cea8b02307dd9 WatchSource:0}: Error finding container 73b90fe0f953c0e004e27aed5656dd4581efdd66a20c0a5db52cea8b02307dd9: Status 404 returned error can't find the container with id 73b90fe0f953c0e004e27aed5656dd4581efdd66a20c0a5db52cea8b02307dd9 Jul 24 12:57:35.850731 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.850696 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:35.850822 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.850765 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:35.850822 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:35.850792 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:36.216287 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:36.216249 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerStarted","Data":"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8"} Jul 24 12:57:36.216489 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:36.216302 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerStarted","Data":"73b90fe0f953c0e004e27aed5656dd4581efdd66a20c0a5db52cea8b02307dd9"} Jul 24 12:57:36.237831 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:36.237782 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podStartSLOduration=1.237766043 podStartE2EDuration="1.237766043s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:57:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:57:36.235444021 +0000 UTC m=+2312.520989723" watchObservedRunningTime="2026-07-24 12:57:36.237766043 +0000 UTC m=+2312.523311727" Jul 24 12:57:36.704365 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:36.704241 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:36.709836 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:36.709807 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:37.221538 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:37.221494 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:37.222409 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:37.222389 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:45.217378 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:45.217337 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 12:57:45.217795 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:45.217419 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 12:57:46.272493 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.272459 2578 generic.go:356] "Generic (PLEG): container finished" podID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerID="7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec" exitCode=0 Jul 24 12:57:46.272912 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.272523 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerDied","Data":"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec"} Jul 24 12:57:46.272969 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.272950 2578 scope.go:117] "RemoveContainer" containerID="7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec" Jul 24 12:57:46.274740 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.274683 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:46.274881 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.274856 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:46.274972 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:46.274955 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:47.279909 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:47.279869 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerStarted","Data":"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9"} Jul 24 12:57:47.583597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:47.583488 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:47.583809 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:47.583781 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 24 12:57:47.583892 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:47.583846 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 24 12:57:48.584225 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:48.584151 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 24 12:57:48.584648 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:48.584242 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 24 12:57:49.583450 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:49.583412 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 24 12:57:49.583630 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:49.583463 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 24 12:57:50.584029 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:50.583992 2578 patch_prober.go:28] interesting pod/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" start-of-body= Jul 24 12:57:50.584434 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:50.584050 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.65:15021/healthz/ready\": dial tcp 10.133.0.65:15021: connect: connection refused" Jul 24 12:57:51.588091 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:51.588050 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:51.588565 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:51.588395 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:51.589294 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:51.589275 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 12:57:53.189621 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.188947 2578 patch_prober.go:28] interesting pod/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:59690->10.133.0.66:15021: read: connection reset by peer" start-of-body= Jul 24 12:57:53.189621 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.189027 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": read tcp 10.133.0.2:59690->10.133.0.66:15021: read: connection reset by peer" Jul 24 12:57:53.312523 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.312489 2578 generic.go:356] "Generic (PLEG): container finished" podID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerID="a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8" exitCode=0 Jul 24 12:57:53.312704 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.312581 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerDied","Data":"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8"} Jul 24 12:57:53.313030 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.313014 2578 scope.go:117] "RemoveContainer" containerID="a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8" Jul 24 12:57:53.313995 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.313965 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:53.314095 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.314049 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:53.314154 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:53.314095 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 12:57:54.322832 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:54.322787 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerStarted","Data":"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41"} Jul 24 12:57:54.703813 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:54.703711 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:54.703813 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:54.703772 2578 patch_prober.go:28] interesting pod/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 12:57:54.704062 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:54.703844 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 12:57:55.703901 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:55.703862 2578 patch_prober.go:28] interesting pod/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 12:57:55.704535 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:55.704501 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 12:57:56.703854 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:56.703807 2578 patch_prober.go:28] interesting pod/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 12:57:56.704070 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:56.703889 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 12:57:57.704297 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:57.704253 2578 patch_prober.go:28] interesting pod/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body= Jul 24 12:57:57.704925 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:57.704891 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" Jul 24 12:57:58.706907 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:58.706872 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:58.707365 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:58.707218 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:57:58.707860 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:57:58.707843 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 12:58:26.730064 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.730029 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 12:58:26.734597 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.734578 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.736810 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.736786 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"kube-root-ca.crt\"" Jul 24 12:58:26.737262 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.737238 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"default-dockercfg-hk67r\"" Jul 24 12:58:26.737366 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.737265 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"openshift-service-ca.crt\"" Jul 24 12:58:26.737366 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.737271 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"auth-test-model-c6697a-kserve-self-signed-certs\"" Jul 24 12:58:26.743232 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.743182 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 12:58:26.854725 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854690 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vw4h\" (UniqueName: \"kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.854916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854733 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.854916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854762 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.854916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854847 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.854916 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854893 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.855085 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.854950 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955507 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vw4h\" (UniqueName: \"kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955555 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955684 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955729 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955759 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955820 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.955992 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.955953 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.956140 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.956116 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.956180 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.956154 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.958246 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.958218 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.958418 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.958399 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:26.963565 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:26.963538 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vw4h\" (UniqueName: \"kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h\") pod \"auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:27.045791 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:27.045751 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:27.182307 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:27.182280 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 12:58:27.184570 ip-10-0-132-126 kubenswrapper[2578]: W0724 12:58:27.184524 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod58286eb7_12b6_46ee_a728_7c7685dd8170.slice/crio-ce8a3581d6cb00167271db29ddc906d59fe573903f33b9a52c51e535ec320dcf WatchSource:0}: Error finding container ce8a3581d6cb00167271db29ddc906d59fe573903f33b9a52c51e535ec320dcf: Status 404 returned error can't find the container with id ce8a3581d6cb00167271db29ddc906d59fe573903f33b9a52c51e535ec320dcf Jul 24 12:58:27.504881 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:27.504775 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" event={"ID":"58286eb7-12b6-46ee-a728-7c7685dd8170","Type":"ContainerStarted","Data":"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e"} Jul 24 12:58:27.504881 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:27.504823 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" event={"ID":"58286eb7-12b6-46ee-a728-7c7685dd8170","Type":"ContainerStarted","Data":"ce8a3581d6cb00167271db29ddc906d59fe573903f33b9a52c51e535ec320dcf"} Jul 24 12:58:27.521179 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:27.520850 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" podStartSLOduration=1.520834887 podStartE2EDuration="1.520834887s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 12:58:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 12:58:27.520649056 +0000 UTC m=+2363.806194741" watchObservedRunningTime="2026-07-24 12:58:27.520834887 +0000 UTC m=+2363.806380572" Jul 24 12:58:37.046550 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:37.046511 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:37.047002 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:37.046584 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:37.058869 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:37.058841 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 12:58:37.567385 ip-10-0-132-126 kubenswrapper[2578]: I0724 12:58:37.567357 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 13:00:34.433347 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:34.433310 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 13:00:49.452927 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.452876 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" podUID="58286eb7-12b6-46ee-a728-7c7685dd8170" containerName="main" containerID="cri-o://afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e" gracePeriod=45 Jul 24 13:00:49.710354 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.710293 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 13:00:49.856435 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856402 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vw4h\" (UniqueName: \"kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856435 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856440 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856686 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856498 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856686 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856516 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856686 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856540 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856686 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856567 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home\") pod \"58286eb7-12b6-46ee-a728-7c7685dd8170\" (UID: \"58286eb7-12b6-46ee-a728-7c7685dd8170\") " Jul 24 13:00:49.856888 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856772 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:00:49.856888 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856815 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache" (OuterVolumeSpecName: "model-cache") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:00:49.856998 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856900 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home" (OuterVolumeSpecName: "home") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:00:49.856998 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.856909 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-tmp-dir\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:49.858873 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.858848 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:00:49.858970 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.858849 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm" (OuterVolumeSpecName: "dshm") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:00:49.858970 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.858892 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h" (OuterVolumeSpecName: "kube-api-access-5vw4h") pod "58286eb7-12b6-46ee-a728-7c7685dd8170" (UID: "58286eb7-12b6-46ee-a728-7c7685dd8170"). InnerVolumeSpecName "kube-api-access-5vw4h". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:00:49.957642 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.957604 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vw4h\" (UniqueName: \"kubernetes.io/projected/58286eb7-12b6-46ee-a728-7c7685dd8170-kube-api-access-5vw4h\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:49.957642 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.957637 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-dshm\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:49.957857 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.957652 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/58286eb7-12b6-46ee-a728-7c7685dd8170-tls-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:49.957857 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.957665 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-model-cache\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:49.957857 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:49.957674 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/58286eb7-12b6-46ee-a728-7c7685dd8170-home\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:00:50.262879 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.262848 2578 generic.go:356] "Generic (PLEG): container finished" podID="58286eb7-12b6-46ee-a728-7c7685dd8170" containerID="afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e" exitCode=0 Jul 24 13:00:50.263028 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.262896 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" Jul 24 13:00:50.263028 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.262933 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" event={"ID":"58286eb7-12b6-46ee-a728-7c7685dd8170","Type":"ContainerDied","Data":"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e"} Jul 24 13:00:50.263028 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.262974 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj" event={"ID":"58286eb7-12b6-46ee-a728-7c7685dd8170","Type":"ContainerDied","Data":"ce8a3581d6cb00167271db29ddc906d59fe573903f33b9a52c51e535ec320dcf"} Jul 24 13:00:50.263028 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.262992 2578 scope.go:117] "RemoveContainer" containerID="afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e" Jul 24 13:00:50.272032 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.272016 2578 scope.go:117] "RemoveContainer" containerID="afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e" Jul 24 13:00:50.272321 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:00:50.272300 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e\": container with ID starting with afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e not found: ID does not exist" containerID="afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e" Jul 24 13:00:50.272400 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.272331 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e"} err="failed to get container status \"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e\": rpc error: code = NotFound desc = could not find container \"afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e\": container with ID starting with afcbaf9c11bff74a26c85223961707fa158f329c5da48bb01272536046f7cc4e not found: ID does not exist" Jul 24 13:00:50.279353 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.279328 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 13:00:50.290959 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:50.290926 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/auth-test-model-c6697a-kserve-7db76fd4b7-s4mlj"] Jul 24 13:00:52.148599 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:00:52.148565 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58286eb7-12b6-46ee-a728-7c7685dd8170" path="/var/lib/kubelet/pods/58286eb7-12b6-46ee-a728-7c7685dd8170/volumes" Jul 24 13:01:20.723878 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.723843 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:01:20.724346 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.724224 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58286eb7-12b6-46ee-a728-7c7685dd8170" containerName="main" Jul 24 13:01:20.724346 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.724235 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="58286eb7-12b6-46ee-a728-7c7685dd8170" containerName="main" Jul 24 13:01:20.724346 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.724293 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="58286eb7-12b6-46ee-a728-7c7685dd8170" containerName="main" Jul 24 13:01:20.728671 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.728652 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.731299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.731168 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"openshift-service-ca.crt\"" Jul 24 13:01:20.731299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.731245 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"kube-root-ca.crt\"" Jul 24 13:01:20.731299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.731266 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"default-dockercfg-hk67r\"" Jul 24 13:01:20.731299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.731269 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-c6697a\"/\"sub-test-model-c6697a-kserve-self-signed-certs\"" Jul 24 13:01:20.744271 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.739669 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:01:20.820683 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820638 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.820683 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820687 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dmvhr\" (UniqueName: \"kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.820916 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820808 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.820916 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820843 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.820916 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820870 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.820916 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.820900 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922005 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.921962 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922005 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922006 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922040 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922058 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dmvhr\" (UniqueName: \"kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922171 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922250 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922542 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922516 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922605 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922549 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.922605 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.922568 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.924668 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.924641 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.924806 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.924787 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:20.930730 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:20.930710 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dmvhr\" (UniqueName: \"kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr\") pod \"sub-test-model-c6697a-kserve-6874b658c9-2f29z\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:21.039971 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:21.039929 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:21.178137 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:21.178107 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:01:21.180016 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:01:21.179987 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2fa44d26_9672_44c4_a1d2_8ccbed57c510.slice/crio-0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910 WatchSource:0}: Error finding container 0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910: Status 404 returned error can't find the container with id 0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910 Jul 24 13:01:21.437409 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:21.437306 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" event={"ID":"2fa44d26-9672-44c4-a1d2-8ccbed57c510","Type":"ContainerStarted","Data":"dccbc3e91d29c03459bc54827a9b2d38dcd95d7e30104a59f896ada21dfa7a2b"} Jul 24 13:01:21.437409 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:21.437358 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" event={"ID":"2fa44d26-9672-44c4-a1d2-8ccbed57c510","Type":"ContainerStarted","Data":"0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910"} Jul 24 13:01:21.456528 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:21.456479 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" podStartSLOduration=1.456461765 podStartE2EDuration="1.456461765s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:01:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:01:21.455309269 +0000 UTC m=+2537.740854950" watchObservedRunningTime="2026-07-24 13:01:21.456461765 +0000 UTC m=+2537.742007449" Jul 24 13:01:31.040486 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:31.040435 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:31.040958 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:31.040500 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:31.053085 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:31.053055 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:31.501016 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:31.500931 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:01:57.457780 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:01:57.457743 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:02:12.476730 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.476686 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" podUID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" containerName="main" containerID="cri-o://dccbc3e91d29c03459bc54827a9b2d38dcd95d7e30104a59f896ada21dfa7a2b" gracePeriod=45 Jul 24 13:02:12.715157 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.715127 2578 generic.go:356] "Generic (PLEG): container finished" podID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" containerID="dccbc3e91d29c03459bc54827a9b2d38dcd95d7e30104a59f896ada21dfa7a2b" exitCode=0 Jul 24 13:02:12.715310 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.715209 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" event={"ID":"2fa44d26-9672-44c4-a1d2-8ccbed57c510","Type":"ContainerDied","Data":"dccbc3e91d29c03459bc54827a9b2d38dcd95d7e30104a59f896ada21dfa7a2b"} Jul 24 13:02:12.715310 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.715250 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" event={"ID":"2fa44d26-9672-44c4-a1d2-8ccbed57c510","Type":"ContainerDied","Data":"0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910"} Jul 24 13:02:12.715310 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.715261 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0287144e73e5f52ae1476949c27e8d1b73a99a053541fceaeff42b791b6d6910" Jul 24 13:02:12.723539 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.723517 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:02:12.820057 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820011 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820057 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820062 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820355 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820086 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820355 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820101 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820355 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820182 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dmvhr\" (UniqueName: \"kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820355 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820237 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home\") pod \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\" (UID: \"2fa44d26-9672-44c4-a1d2-8ccbed57c510\") " Jul 24 13:02:12.820550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820390 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:02:12.820550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820434 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache" (OuterVolumeSpecName: "model-cache") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:02:12.820550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820520 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home" (OuterVolumeSpecName: "home") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:02:12.820660 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820559 2578 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tmp-dir\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:12.820660 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.820581 2578 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-model-cache\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:12.822445 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.822420 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm" (OuterVolumeSpecName: "dshm") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:02:12.822536 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.822478 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 24 13:02:12.822574 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.822544 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr" (OuterVolumeSpecName: "kube-api-access-dmvhr") pod "2fa44d26-9672-44c4-a1d2-8ccbed57c510" (UID: "2fa44d26-9672-44c4-a1d2-8ccbed57c510"). InnerVolumeSpecName "kube-api-access-dmvhr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:02:12.921837 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.921788 2578 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fa44d26-9672-44c4-a1d2-8ccbed57c510-tls-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:12.921837 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.921831 2578 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-dshm\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:12.921837 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.921840 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dmvhr\" (UniqueName: \"kubernetes.io/projected/2fa44d26-9672-44c4-a1d2-8ccbed57c510-kube-api-access-dmvhr\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:12.921837 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:12.921848 2578 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/2fa44d26-9672-44c4-a1d2-8ccbed57c510-home\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:02:13.721072 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:13.721040 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z" Jul 24 13:02:13.746407 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:13.746378 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:02:13.749771 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:13.749744 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-c6697a/sub-test-model-c6697a-kserve-6874b658c9-2f29z"] Jul 24 13:02:14.148187 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:14.148147 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" path="/var/lib/kubelet/pods/2fa44d26-9672-44c4-a1d2-8ccbed57c510/volumes" Jul 24 13:02:17.370258 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:02:17.370224 2578 scope.go:117] "RemoveContainer" containerID="abbfead1b207be0f97c4fe8830ac268859568dddfdaa24008f2e87ef8802d47d" Jul 24 13:03:33.389329 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.389295 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7904-140605933071664"] Jul 24 13:03:33.389806 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.389651 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" containerName="main" Jul 24 13:03:33.389806 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.389663 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" containerName="main" Jul 24 13:03:33.389806 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.389724 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2fa44d26-9672-44c4-a1d2-8ccbed57c510" containerName="main" Jul 24 13:03:33.392801 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.392784 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:33.395212 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.395165 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-xg2cx\"" Jul 24 13:03:33.398610 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.398585 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933071664"] Jul 24 13:03:33.460006 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.459968 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lpdc6\" (UniqueName: \"kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6\") pod \"test-curl-7904-140605933071664\" (UID: \"02bbd92e-1320-46e1-a17b-38932b1a60e8\") " pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:33.560831 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.560793 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lpdc6\" (UniqueName: \"kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6\") pod \"test-curl-7904-140605933071664\" (UID: \"02bbd92e-1320-46e1-a17b-38932b1a60e8\") " pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:33.569614 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.569591 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lpdc6\" (UniqueName: \"kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6\") pod \"test-curl-7904-140605933071664\" (UID: \"02bbd92e-1320-46e1-a17b-38932b1a60e8\") " pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:33.703060 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.702965 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:33.842440 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.842411 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933071664"] Jul 24 13:03:33.844286 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:33.844248 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod02bbd92e_1320_46e1_a17b_38932b1a60e8.slice/crio-195782cddf82a945a519ebadf242d78873f458e0ae9d1d5f96be3f85dbb4b755 WatchSource:0}: Error finding container 195782cddf82a945a519ebadf242d78873f458e0ae9d1d5f96be3f85dbb4b755: Status 404 returned error can't find the container with id 195782cddf82a945a519ebadf242d78873f458e0ae9d1d5f96be3f85dbb4b755 Jul 24 13:03:33.845604 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:33.845583 2578 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 24 13:03:34.148432 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:34.148397 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933071664" event={"ID":"02bbd92e-1320-46e1-a17b-38932b1a60e8","Type":"ContainerStarted","Data":"195782cddf82a945a519ebadf242d78873f458e0ae9d1d5f96be3f85dbb4b755"} Jul 24 13:03:35.149833 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:35.149795 2578 generic.go:356] "Generic (PLEG): container finished" podID="02bbd92e-1320-46e1-a17b-38932b1a60e8" containerID="506330f8be16ac4dff16b80c6a9e1ae7e517818812f06026ec07071274233685" exitCode=0 Jul 24 13:03:35.150302 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:35.149883 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933071664" event={"ID":"02bbd92e-1320-46e1-a17b-38932b1a60e8","Type":"ContainerDied","Data":"506330f8be16ac4dff16b80c6a9e1ae7e517818812f06026ec07071274233685"} Jul 24 13:03:36.321130 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.321097 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:36.357795 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.357767 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7904-140605933071664_02bbd92e-1320-46e1-a17b-38932b1a60e8/test-curl-7904-140605933071664/0.log" Jul 24 13:03:36.382851 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.382815 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7904-140605933071664"] Jul 24 13:03:36.386923 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.386899 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7904-140605933071664"] Jul 24 13:03:36.390529 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.390512 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lpdc6\" (UniqueName: \"kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6\") pod \"02bbd92e-1320-46e1-a17b-38932b1a60e8\" (UID: \"02bbd92e-1320-46e1-a17b-38932b1a60e8\") " Jul 24 13:03:36.392763 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.392743 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6" (OuterVolumeSpecName: "kube-api-access-lpdc6") pod "02bbd92e-1320-46e1-a17b-38932b1a60e8" (UID: "02bbd92e-1320-46e1-a17b-38932b1a60e8"). InnerVolumeSpecName "kube-api-access-lpdc6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:03:36.491629 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.491541 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lpdc6\" (UniqueName: \"kubernetes.io/projected/02bbd92e-1320-46e1-a17b-38932b1a60e8-kube-api-access-lpdc6\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:03:36.513993 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.513961 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7904-140605933069488"] Jul 24 13:03:36.514334 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.514321 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="02bbd92e-1320-46e1-a17b-38932b1a60e8" containerName="test-curl-7904-140605933071664" Jul 24 13:03:36.514398 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.514336 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="02bbd92e-1320-46e1-a17b-38932b1a60e8" containerName="test-curl-7904-140605933071664" Jul 24 13:03:36.514444 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.514417 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="02bbd92e-1320-46e1-a17b-38932b1a60e8" containerName="test-curl-7904-140605933071664" Jul 24 13:03:36.517370 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.517343 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:36.521610 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.521588 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933069488"] Jul 24 13:03:36.592512 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.592480 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2r2m\" (UniqueName: \"kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m\") pod \"test-curl-7904-140605933069488\" (UID: \"9aa0db43-7ecd-4797-b5bb-28d2b783f75a\") " pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:36.693666 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.693626 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2r2m\" (UniqueName: \"kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m\") pod \"test-curl-7904-140605933069488\" (UID: \"9aa0db43-7ecd-4797-b5bb-28d2b783f75a\") " pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:36.702268 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.702239 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2r2m\" (UniqueName: \"kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m\") pod \"test-curl-7904-140605933069488\" (UID: \"9aa0db43-7ecd-4797-b5bb-28d2b783f75a\") " pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:36.829382 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:36.829346 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:37.161815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:37.161742 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:37.161957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:37.161744 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="195782cddf82a945a519ebadf242d78873f458e0ae9d1d5f96be3f85dbb4b755" Jul 24 13:03:37.171378 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:03:37.171346 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7904-140605933071664\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="02bbd92e-1320-46e1-a17b-38932b1a60e8" pod="opendatahub/test-curl-7904-140605933071664" Jul 24 13:03:37.173136 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:37.173116 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933069488"] Jul 24 13:03:37.175001 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:37.174972 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9aa0db43_7ecd_4797_b5bb_28d2b783f75a.slice/crio-a801d3db7946a12b926d232da1c280c5d4758c27ec108ef676e424b2de8ba7cf WatchSource:0}: Error finding container a801d3db7946a12b926d232da1c280c5d4758c27ec108ef676e424b2de8ba7cf: Status 404 returned error can't find the container with id a801d3db7946a12b926d232da1c280c5d4758c27ec108ef676e424b2de8ba7cf Jul 24 13:03:38.148031 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:38.147999 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="02bbd92e-1320-46e1-a17b-38932b1a60e8" path="/var/lib/kubelet/pods/02bbd92e-1320-46e1-a17b-38932b1a60e8/volumes" Jul 24 13:03:38.167957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:38.167925 2578 generic.go:356] "Generic (PLEG): container finished" podID="9aa0db43-7ecd-4797-b5bb-28d2b783f75a" containerID="7ac5ea0dbfaa2d9f4536ce28fc12df7acdb8910433a348e4a292e63eb6a32c2f" exitCode=0 Jul 24 13:03:38.168126 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:38.168020 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933069488" event={"ID":"9aa0db43-7ecd-4797-b5bb-28d2b783f75a","Type":"ContainerDied","Data":"7ac5ea0dbfaa2d9f4536ce28fc12df7acdb8910433a348e4a292e63eb6a32c2f"} Jul 24 13:03:38.168126 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:38.168064 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933069488" event={"ID":"9aa0db43-7ecd-4797-b5bb-28d2b783f75a","Type":"ContainerStarted","Data":"a801d3db7946a12b926d232da1c280c5d4758c27ec108ef676e424b2de8ba7cf"} Jul 24 13:03:39.307281 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.307251 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:39.326892 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.326861 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7904-140605933069488_9aa0db43-7ecd-4797-b5bb-28d2b783f75a/test-curl-7904-140605933069488/0.log" Jul 24 13:03:39.347711 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.347678 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7904-140605933069488"] Jul 24 13:03:39.351954 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.351926 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7904-140605933069488"] Jul 24 13:03:39.418785 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.418751 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q2r2m\" (UniqueName: \"kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m\") pod \"9aa0db43-7ecd-4797-b5bb-28d2b783f75a\" (UID: \"9aa0db43-7ecd-4797-b5bb-28d2b783f75a\") " Jul 24 13:03:39.420931 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.420909 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m" (OuterVolumeSpecName: "kube-api-access-q2r2m") pod "9aa0db43-7ecd-4797-b5bb-28d2b783f75a" (UID: "9aa0db43-7ecd-4797-b5bb-28d2b783f75a"). InnerVolumeSpecName "kube-api-access-q2r2m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:03:39.509777 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.509741 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:39.510084 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.510072 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9aa0db43-7ecd-4797-b5bb-28d2b783f75a" containerName="test-curl-7904-140605933069488" Jul 24 13:03:39.510127 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.510086 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="9aa0db43-7ecd-4797-b5bb-28d2b783f75a" containerName="test-curl-7904-140605933069488" Jul 24 13:03:39.510161 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.510148 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="9aa0db43-7ecd-4797-b5bb-28d2b783f75a" containerName="test-curl-7904-140605933069488" Jul 24 13:03:39.512988 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.512972 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:39.517483 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.517459 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:39.519479 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.519457 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q2r2m\" (UniqueName: \"kubernetes.io/projected/9aa0db43-7ecd-4797-b5bb-28d2b783f75a-kube-api-access-q2r2m\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:03:39.620814 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.620774 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55vdh\" (UniqueName: \"kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh\") pod \"test-curl-7904-140605927700496\" (UID: \"4371198e-e1e3-40b3-91e1-097d81cadb9b\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:39.721843 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.721751 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55vdh\" (UniqueName: \"kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh\") pod \"test-curl-7904-140605927700496\" (UID: \"4371198e-e1e3-40b3-91e1-097d81cadb9b\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:39.730749 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.730718 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55vdh\" (UniqueName: \"kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh\") pod \"test-curl-7904-140605927700496\" (UID: \"4371198e-e1e3-40b3-91e1-097d81cadb9b\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:39.823354 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.823322 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:39.956233 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:39.956185 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:39.958451 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:39.958423 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4371198e_e1e3_40b3_91e1_097d81cadb9b.slice/crio-cdb0fe3b594899ddc096e2b286ea9ffadbd071a3f9a4b7435284455943a25894 WatchSource:0}: Error finding container cdb0fe3b594899ddc096e2b286ea9ffadbd071a3f9a4b7435284455943a25894: Status 404 returned error can't find the container with id cdb0fe3b594899ddc096e2b286ea9ffadbd071a3f9a4b7435284455943a25894 Jul 24 13:03:40.152746 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:40.152710 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9aa0db43-7ecd-4797-b5bb-28d2b783f75a" path="/var/lib/kubelet/pods/9aa0db43-7ecd-4797-b5bb-28d2b783f75a/volumes" Jul 24 13:03:40.179503 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:40.179468 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605927700496" event={"ID":"4371198e-e1e3-40b3-91e1-097d81cadb9b","Type":"ContainerStarted","Data":"cdb0fe3b594899ddc096e2b286ea9ffadbd071a3f9a4b7435284455943a25894"} Jul 24 13:03:40.180710 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:40.180691 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933069488" Jul 24 13:03:40.181277 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:40.180683 2578 scope.go:117] "RemoveContainer" containerID="7ac5ea0dbfaa2d9f4536ce28fc12df7acdb8910433a348e4a292e63eb6a32c2f" Jul 24 13:03:41.188487 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:41.188457 2578 generic.go:356] "Generic (PLEG): container finished" podID="4371198e-e1e3-40b3-91e1-097d81cadb9b" containerID="739ab2d77cf8e368a09fd995759240b09bf8f30afa3672f457c07c75476fc863" exitCode=0 Jul 24 13:03:41.188935 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:41.188513 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605927700496" event={"ID":"4371198e-e1e3-40b3-91e1-097d81cadb9b","Type":"ContainerDied","Data":"739ab2d77cf8e368a09fd995759240b09bf8f30afa3672f457c07c75476fc863"} Jul 24 13:03:42.326799 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.326771 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.345846 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.345819 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7904-140605927700496_4371198e-e1e3-40b3-91e1-097d81cadb9b/test-curl-7904-140605927700496/0.log" Jul 24 13:03:42.367740 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.367670 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:42.369859 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.369838 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:42.443632 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.443593 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-55vdh\" (UniqueName: \"kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh\") pod \"4371198e-e1e3-40b3-91e1-097d81cadb9b\" (UID: \"4371198e-e1e3-40b3-91e1-097d81cadb9b\") " Jul 24 13:03:42.445902 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.445838 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh" (OuterVolumeSpecName: "kube-api-access-55vdh") pod "4371198e-e1e3-40b3-91e1-097d81cadb9b" (UID: "4371198e-e1e3-40b3-91e1-097d81cadb9b"). InnerVolumeSpecName "kube-api-access-55vdh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:03:42.489874 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.489833 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:42.490262 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.490244 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4371198e-e1e3-40b3-91e1-097d81cadb9b" containerName="test-curl-7904-140605927700496" Jul 24 13:03:42.490372 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.490263 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4371198e-e1e3-40b3-91e1-097d81cadb9b" containerName="test-curl-7904-140605927700496" Jul 24 13:03:42.490372 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.490355 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4371198e-e1e3-40b3-91e1-097d81cadb9b" containerName="test-curl-7904-140605927700496" Jul 24 13:03:42.493316 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.493296 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.500611 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.500589 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:42.545149 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.545113 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-55vdh\" (UniqueName: \"kubernetes.io/projected/4371198e-e1e3-40b3-91e1-097d81cadb9b-kube-api-access-55vdh\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:03:42.646309 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.646270 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjp5g\" (UniqueName: \"kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g\") pod \"test-curl-7904-140605927700496\" (UID: \"72830ee3-75a8-487b-a67a-7b20628ce97e\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.747803 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.747699 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kjp5g\" (UniqueName: \"kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g\") pod \"test-curl-7904-140605927700496\" (UID: \"72830ee3-75a8-487b-a67a-7b20628ce97e\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.756411 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.756379 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjp5g\" (UniqueName: \"kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g\") pod \"test-curl-7904-140605927700496\" (UID: \"72830ee3-75a8-487b-a67a-7b20628ce97e\") " pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.802714 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.802672 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:42.936014 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:42.935988 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:42.937982 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:42.937951 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod72830ee3_75a8_487b_a67a_7b20628ce97e.slice/crio-9371d2b311900122dcefa3b59e45fc2f4665c65847bda294ee6229fc38660c5a WatchSource:0}: Error finding container 9371d2b311900122dcefa3b59e45fc2f4665c65847bda294ee6229fc38660c5a: Status 404 returned error can't find the container with id 9371d2b311900122dcefa3b59e45fc2f4665c65847bda294ee6229fc38660c5a Jul 24 13:03:43.201055 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:43.201024 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:43.201239 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:43.201027 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cdb0fe3b594899ddc096e2b286ea9ffadbd071a3f9a4b7435284455943a25894" Jul 24 13:03:43.202241 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:43.202212 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605927700496" event={"ID":"72830ee3-75a8-487b-a67a-7b20628ce97e","Type":"ContainerStarted","Data":"9371d2b311900122dcefa3b59e45fc2f4665c65847bda294ee6229fc38660c5a"} Jul 24 13:03:43.212455 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:43.212424 2578 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="opendatahub/test-curl-7904-140605927700496" oldPodUID="4371198e-e1e3-40b3-91e1-097d81cadb9b" podUID="72830ee3-75a8-487b-a67a-7b20628ce97e" Jul 24 13:03:44.148035 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:44.147996 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4371198e-e1e3-40b3-91e1-097d81cadb9b" path="/var/lib/kubelet/pods/4371198e-e1e3-40b3-91e1-097d81cadb9b/volumes" Jul 24 13:03:44.209377 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:44.209344 2578 generic.go:356] "Generic (PLEG): container finished" podID="72830ee3-75a8-487b-a67a-7b20628ce97e" containerID="43537b61f4bb25d8ce030fd295f728d38f21849325d03772cde36b0c27b4a976" exitCode=0 Jul 24 13:03:44.209551 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:44.209431 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605927700496" event={"ID":"72830ee3-75a8-487b-a67a-7b20628ce97e","Type":"ContainerDied","Data":"43537b61f4bb25d8ce030fd295f728d38f21849325d03772cde36b0c27b4a976"} Jul 24 13:03:45.344932 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.344908 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:45.364659 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.364609 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7904-140605927700496_72830ee3-75a8-487b-a67a-7b20628ce97e/test-curl-7904-140605927700496/0.log" Jul 24 13:03:45.386833 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.386801 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:45.389973 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.389947 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7904-140605927700496"] Jul 24 13:03:45.471948 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.471860 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kjp5g\" (UniqueName: \"kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g\") pod \"72830ee3-75a8-487b-a67a-7b20628ce97e\" (UID: \"72830ee3-75a8-487b-a67a-7b20628ce97e\") " Jul 24 13:03:45.474042 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.474019 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g" (OuterVolumeSpecName: "kube-api-access-kjp5g") pod "72830ee3-75a8-487b-a67a-7b20628ce97e" (UID: "72830ee3-75a8-487b-a67a-7b20628ce97e"). InnerVolumeSpecName "kube-api-access-kjp5g". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:03:45.518299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.518264 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7904-140605933188560"] Jul 24 13:03:45.518664 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.518649 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="72830ee3-75a8-487b-a67a-7b20628ce97e" containerName="test-curl-7904-140605927700496" Jul 24 13:03:45.518758 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.518666 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="72830ee3-75a8-487b-a67a-7b20628ce97e" containerName="test-curl-7904-140605927700496" Jul 24 13:03:45.518758 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.518749 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="72830ee3-75a8-487b-a67a-7b20628ce97e" containerName="test-curl-7904-140605927700496" Jul 24 13:03:45.521764 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.521745 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:45.525228 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.525180 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933188560"] Jul 24 13:03:45.573365 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.573326 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kjp5g\" (UniqueName: \"kubernetes.io/projected/72830ee3-75a8-487b-a67a-7b20628ce97e-kube-api-access-kjp5g\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:03:45.674554 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.674507 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7mn5\" (UniqueName: \"kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5\") pod \"test-curl-7904-140605933188560\" (UID: \"4916464a-945d-4ec3-a930-db390bbdf07e\") " pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:45.775835 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.775794 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7mn5\" (UniqueName: \"kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5\") pod \"test-curl-7904-140605933188560\" (UID: \"4916464a-945d-4ec3-a930-db390bbdf07e\") " pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:45.785132 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.785106 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7mn5\" (UniqueName: \"kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5\") pod \"test-curl-7904-140605933188560\" (UID: \"4916464a-945d-4ec3-a930-db390bbdf07e\") " pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:45.833108 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.833071 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:45.965467 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:45.965409 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7904-140605933188560"] Jul 24 13:03:45.967609 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:45.967580 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4916464a_945d_4ec3_a930_db390bbdf07e.slice/crio-133c1590d0bc11db1edba212f9cbd6b65e5b16b356518f145ed1eab9733b0f02 WatchSource:0}: Error finding container 133c1590d0bc11db1edba212f9cbd6b65e5b16b356518f145ed1eab9733b0f02: Status 404 returned error can't find the container with id 133c1590d0bc11db1edba212f9cbd6b65e5b16b356518f145ed1eab9733b0f02 Jul 24 13:03:46.149093 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:46.149013 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="72830ee3-75a8-487b-a67a-7b20628ce97e" path="/var/lib/kubelet/pods/72830ee3-75a8-487b-a67a-7b20628ce97e/volumes" Jul 24 13:03:46.221181 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:46.221155 2578 scope.go:117] "RemoveContainer" containerID="43537b61f4bb25d8ce030fd295f728d38f21849325d03772cde36b0c27b4a976" Jul 24 13:03:46.221393 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:46.221155 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605927700496" Jul 24 13:03:46.222464 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:46.222402 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933188560" event={"ID":"4916464a-945d-4ec3-a930-db390bbdf07e","Type":"ContainerStarted","Data":"133c1590d0bc11db1edba212f9cbd6b65e5b16b356518f145ed1eab9733b0f02"} Jul 24 13:03:47.230123 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:47.230088 2578 generic.go:356] "Generic (PLEG): container finished" podID="4916464a-945d-4ec3-a930-db390bbdf07e" containerID="1c203fe67498480f0447439f744bf8aad2c9cf9680f3cf452580d5b9c1bc31ee" exitCode=0 Jul 24 13:03:47.230618 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:47.230177 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7904-140605933188560" event={"ID":"4916464a-945d-4ec3-a930-db390bbdf07e","Type":"ContainerDied","Data":"1c203fe67498480f0447439f744bf8aad2c9cf9680f3cf452580d5b9c1bc31ee"} Jul 24 13:03:48.380932 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.380904 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:48.403310 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.403284 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_test-curl-7904-140605933188560_4916464a-945d-4ec3-a930-db390bbdf07e/test-curl-7904-140605933188560/0.log" Jul 24 13:03:48.426904 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.426869 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7904-140605933188560"] Jul 24 13:03:48.430019 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.429991 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7904-140605933188560"] Jul 24 13:03:48.499668 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.499582 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d7mn5\" (UniqueName: \"kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5\") pod \"4916464a-945d-4ec3-a930-db390bbdf07e\" (UID: \"4916464a-945d-4ec3-a930-db390bbdf07e\") " Jul 24 13:03:48.501972 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.501949 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5" (OuterVolumeSpecName: "kube-api-access-d7mn5") pod "4916464a-945d-4ec3-a930-db390bbdf07e" (UID: "4916464a-945d-4ec3-a930-db390bbdf07e"). InnerVolumeSpecName "kube-api-access-d7mn5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:03:48.600730 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:48.600675 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d7mn5\" (UniqueName: \"kubernetes.io/projected/4916464a-945d-4ec3-a930-db390bbdf07e-kube-api-access-d7mn5\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:03:49.247835 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:49.247806 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:49.248023 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:49.247841 2578 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="133c1590d0bc11db1edba212f9cbd6b65e5b16b356518f145ed1eab9733b0f02" Jul 24 13:03:49.257608 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:03:49.257575 2578 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7904-140605933188560\" is forbidden: User \"system:node:ip-10-0-132-126.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-132-126.ec2.internal' and this object" podUID="4916464a-945d-4ec3-a930-db390bbdf07e" pod="opendatahub/test-curl-7904-140605933188560" Jul 24 13:03:50.148719 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:50.148685 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4916464a-945d-4ec3-a930-db390bbdf07e" path="/var/lib/kubelet/pods/4916464a-945d-4ec3-a930-db390bbdf07e/volumes" Jul 24 13:03:52.783422 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.783381 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:03:52.783874 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.783731 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4916464a-945d-4ec3-a930-db390bbdf07e" containerName="test-curl-7904-140605933188560" Jul 24 13:03:52.783874 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.783741 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4916464a-945d-4ec3-a930-db390bbdf07e" containerName="test-curl-7904-140605933188560" Jul 24 13:03:52.783874 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.783798 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4916464a-945d-4ec3-a930-db390bbdf07e" containerName="test-curl-7904-140605933188560" Jul 24 13:03:52.787003 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.786986 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.789514 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.789497 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-967d3e-openshift-default-dockercfg-4s6ph\"" Jul 24 13:03:52.799665 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.799641 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:03:52.834776 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834746 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.834776 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834779 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834799 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jctw\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834883 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834956 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.834985 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.835001 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835309 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.835026 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.835309 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.835090 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935584 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935550 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935584 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935584 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935603 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935628 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935672 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935696 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935714 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.935815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935804 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jctw\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936083 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.935887 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936083 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.936003 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936240 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.936184 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936313 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.936296 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936377 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.936339 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.936424 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.936407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.938159 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.938136 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.938424 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.938407 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.960348 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.960322 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jctw\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:52.960887 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:52.960870 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token\") pod \"e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:53.097656 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.097564 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:53.244580 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.244554 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:03:53.246272 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:03:53.246245 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod74db8f88_b347_43d4_af48_4e8fe1ad7a5e.slice/crio-55c110654d24ba46919470ddd139461079dac8b5edbd864e75347d288240b584 WatchSource:0}: Error finding container 55c110654d24ba46919470ddd139461079dac8b5edbd864e75347d288240b584: Status 404 returned error can't find the container with id 55c110654d24ba46919470ddd139461079dac8b5edbd864e75347d288240b584 Jul 24 13:03:53.248421 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.248388 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:03:53.248487 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.248452 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:03:53.248525 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.248484 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:03:53.269923 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:53.269892 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerStarted","Data":"55c110654d24ba46919470ddd139461079dac8b5edbd864e75347d288240b584"} Jul 24 13:03:54.277369 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:54.277293 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerStarted","Data":"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441"} Jul 24 13:03:54.298303 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:54.298243 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podStartSLOduration=2.298225934 podStartE2EDuration="2.298225934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:03:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:03:54.297119135 +0000 UTC m=+2690.582664843" watchObservedRunningTime="2026-07-24 13:03:54.298225934 +0000 UTC m=+2690.583771619" Jul 24 13:03:55.098438 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:55.098399 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:55.103515 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:55.103482 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:55.282815 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:55.282786 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:03:55.284007 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:03:55.283984 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:04:11.325373 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:11.325325 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Jul 24 13:04:11.325826 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:11.325401 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Jul 24 13:04:12.065067 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.065031 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:04:12.067812 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.067794 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.070559 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.070532 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-47a7ae-openshift-default-dockercfg-mp496\"" Jul 24 13:04:12.078271 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.078247 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:04:12.105633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105606 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105788 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105637 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpm2j\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105788 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105665 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105788 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105680 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105788 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105760 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105966 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105805 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105966 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105848 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105966 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105867 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.105966 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.105885 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207291 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207188 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207291 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207237 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpm2j\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207291 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207270 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207291 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207285 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207304 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207329 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207369 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207400 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207429 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207841 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207678 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.207841 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.207717 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.208052 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.208032 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.208149 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.208127 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.208490 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.208271 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.209970 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.209950 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.210096 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.210078 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.215544 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.215520 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.215657 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.215643 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpm2j\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j\") pod \"e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.379275 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.379229 2578 generic.go:356] "Generic (PLEG): container finished" podID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerID="bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441" exitCode=0 Jul 24 13:04:12.379675 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.379294 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerDied","Data":"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441"} Jul 24 13:04:12.379765 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.379750 2578 scope.go:117] "RemoveContainer" containerID="bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441" Jul 24 13:04:12.380895 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.380866 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:12.380949 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.380936 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:12.380993 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.380982 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:12.402493 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.402465 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:12.560359 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.560331 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:04:12.562164 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:04:12.562120 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c83c362_8c46_4633_bf95_d055b4074d52.slice/crio-5c7c287cd172849c013fed4442637553d8f65e52812924dd1f938657a3b1df96 WatchSource:0}: Error finding container 5c7c287cd172849c013fed4442637553d8f65e52812924dd1f938657a3b1df96: Status 404 returned error can't find the container with id 5c7c287cd172849c013fed4442637553d8f65e52812924dd1f938657a3b1df96 Jul 24 13:04:12.564537 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.564500 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:12.564613 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.564579 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:12.564654 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:12.564615 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:13.387260 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.387227 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerStarted","Data":"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b"} Jul 24 13:04:13.388780 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.388753 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerStarted","Data":"3b3e4b5c5ef91818d17c2b6ecd6e583f04e6cac7892e10ef8fef8c4cf71a99ff"} Jul 24 13:04:13.388911 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.388799 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerStarted","Data":"5c7c287cd172849c013fed4442637553d8f65e52812924dd1f938657a3b1df96"} Jul 24 13:04:13.403086 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.403038 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:13.413112 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.413071 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:13.430245 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:13.430161 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podStartSLOduration=1.430139738 podStartE2EDuration="1.430139738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:04:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:04:13.429090332 +0000 UTC m=+2709.714636051" watchObservedRunningTime="2026-07-24 13:04:13.430139738 +0000 UTC m=+2709.715685422" Jul 24 13:04:14.098401 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:14.098360 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:04:14.098635 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:14.098607 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:04:14.098712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:14.098685 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 13:04:14.394955 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:14.394857 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:14.395772 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:14.395753 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:15.098784 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:15.098731 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:04:15.099012 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:15.098807 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 13:04:16.098826 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:16.098786 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:04:16.099335 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:16.098860 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 13:04:17.098530 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:17.098480 2578 patch_prober.go:28] interesting pod/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body= Jul 24 13:04:17.098746 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:17.098562 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" Jul 24 13:04:18.102211 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:18.102166 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:04:18.102686 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:18.102558 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:04:18.103142 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:18.103122 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:04:27.490060 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.490026 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c83c362-8c46-4633-bf95-d055b4074d52" containerID="3b3e4b5c5ef91818d17c2b6ecd6e583f04e6cac7892e10ef8fef8c4cf71a99ff" exitCode=0 Jul 24 13:04:27.490517 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.490099 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerDied","Data":"3b3e4b5c5ef91818d17c2b6ecd6e583f04e6cac7892e10ef8fef8c4cf71a99ff"} Jul 24 13:04:27.490587 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.490570 2578 scope.go:117] "RemoveContainer" containerID="3b3e4b5c5ef91818d17c2b6ecd6e583f04e6cac7892e10ef8fef8c4cf71a99ff" Jul 24 13:04:27.491843 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.491813 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:27.491915 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.491903 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:27.491949 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:27.491936 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:04:28.520361 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:28.520300 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerStarted","Data":"61ceae5e40f512748fb9d994957231ea3be982b056e45dea1dc6f0338a3b512d"} Jul 24 13:04:29.395395 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:29.395355 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:29.403615 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:29.403571 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:29.405167 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:29.405131 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 24 13:04:29.405327 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:29.405210 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 24 13:04:30.403621 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:30.403570 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 24 13:04:30.404129 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:30.403662 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 24 13:04:31.403920 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:31.403877 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 24 13:04:31.404566 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:31.404531 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 24 13:04:32.402989 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:32.402942 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" start-of-body= Jul 24 13:04:32.403171 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:32.403013 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.75:15021/healthz/ready\": dial tcp 10.133.0.75:15021: connect: connection refused" Jul 24 13:04:33.407589 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:33.407559 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:04:33.408644 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:04:33.408622 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:05:30.185741 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.185654 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:05:30.189007 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.188989 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.191032 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.190984 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-30955e-openshift-default-dockercfg-gnv6h\"" Jul 24 13:05:30.201155 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.201127 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:05:30.294803 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.294763 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.295491 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.295466 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.296032 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.296011 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.296209 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.296174 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.296418 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.296401 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.296817 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.296798 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqxvs\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.296982 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.296966 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.297295 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.297278 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.297660 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.297643 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399454 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399416 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399466 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399500 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399536 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqxvs\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399576 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399608 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399953 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399681 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399953 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399732 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399953 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399782 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399953 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399847 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.399953 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.399931 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.400275 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.400118 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.400275 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.400247 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.400830 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.400806 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.402088 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.402062 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.402544 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.402511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.408251 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.408226 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.408359 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.408345 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqxvs\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs\") pod \"e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.500821 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.500734 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:30.652014 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.651986 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:05:30.653746 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:05:30.653700 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1178b5fa_2a16_4c34_a222_65d97019e2a0.slice/crio-1bb71b960b195d1ce558746959c2096c258f7da89495e0ec24b7bd1539a756ed WatchSource:0}: Error finding container 1bb71b960b195d1ce558746959c2096c258f7da89495e0ec24b7bd1539a756ed: Status 404 returned error can't find the container with id 1bb71b960b195d1ce558746959c2096c258f7da89495e0ec24b7bd1539a756ed Jul 24 13:05:30.656133 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.656096 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:30.656225 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.656185 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:30.656270 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.656254 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:30.867024 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.866981 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerStarted","Data":"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5"} Jul 24 13:05:30.867246 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.867047 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerStarted","Data":"1bb71b960b195d1ce558746959c2096c258f7da89495e0ec24b7bd1539a756ed"} Jul 24 13:05:30.901393 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:30.901338 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podStartSLOduration=0.901319037 podStartE2EDuration="901.319037ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-24 13:05:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:05:30.899104147 +0000 UTC m=+2787.184649855" watchObservedRunningTime="2026-07-24 13:05:30.901319037 +0000 UTC m=+2787.186864722" Jul 24 13:05:31.501089 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:31.501052 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:31.506516 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:31.506490 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:31.872655 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:31.872624 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:31.873641 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:31.873617 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:46.874480 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.874358 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 13:05:46.874480 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.874415 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 13:05:46.970151 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.970051 2578 generic.go:356] "Generic (PLEG): container finished" podID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerID="861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5" exitCode=0 Jul 24 13:05:46.970151 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.970122 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerDied","Data":"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5"} Jul 24 13:05:46.970651 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.970634 2578 scope.go:117] "RemoveContainer" containerID="861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5" Jul 24 13:05:46.971772 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.971745 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:46.971854 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.971812 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:46.971854 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:46.971844 2578 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892160Ki","pods":"250"} Jul 24 13:05:47.977917 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:47.977881 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerStarted","Data":"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101"} Jul 24 13:05:48.501633 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:48.501589 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:48.501858 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:48.501729 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 13:05:48.501858 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:48.501776 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 13:05:49.502174 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:49.502124 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 13:05:49.502733 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:49.502231 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 13:05:50.501902 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:50.501852 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 13:05:50.502137 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:50.501927 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 13:05:51.501740 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:51.501698 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" start-of-body= Jul 24 13:05:51.502284 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:51.501774 2578 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.76:15021/healthz/ready\": dial tcp 10.133.0.76:15021: connect: connection refused" Jul 24 13:05:52.505585 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:52.505550 2578 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:52.505988 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:52.505928 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:05:52.506646 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:05:52.506627 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:06:20.149657 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:20.149628 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:06:20.150172 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:20.149829 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" containerID="cri-o://20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101" gracePeriod=30 Jul 24 13:06:22.508295 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:22.508251 2578 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused Jul 24 13:06:22.508698 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:22.508340 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:06:25.408095 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.408066 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:06:25.446759 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446657 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.446759 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446704 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.446997 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446766 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.446997 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446815 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.446997 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446862 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.446997 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.446981 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.447240 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447007 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:06:25.447240 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447051 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pqxvs\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.447240 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447086 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.447240 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447114 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs\") pod \"1178b5fa-2a16-4c34-a222-65d97019e2a0\" (UID: \"1178b5fa-2a16-4c34-a222-65d97019e2a0\") " Jul 24 13:06:25.447651 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447456 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.448423 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.447847 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:06:25.448423 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.448338 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:06:25.448423 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.448399 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:06:25.448652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.448491 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data" (OuterVolumeSpecName: "istio-data") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:06:25.449799 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.449762 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:06:25.449799 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.449773 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:06:25.449967 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.449902 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token" (OuterVolumeSpecName: "istio-token") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:06:25.450159 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.450143 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs" (OuterVolumeSpecName: "kube-api-access-pqxvs") pod "1178b5fa-2a16-4c34-a222-65d97019e2a0" (UID: "1178b5fa-2a16-4c34-a222-65d97019e2a0"). InnerVolumeSpecName "kube-api-access-pqxvs". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:06:25.547992 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.547956 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.547992 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.547987 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.547992 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.547999 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1178b5fa-2a16-4c34-a222-65d97019e2a0-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.548253 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.548007 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pqxvs\" (UniqueName: \"kubernetes.io/projected/1178b5fa-2a16-4c34-a222-65d97019e2a0-kube-api-access-pqxvs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.548253 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.548016 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.548253 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.548025 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.548253 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.548034 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:25.548253 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:25.548041 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1178b5fa-2a16-4c34-a222-65d97019e2a0-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:06:26.199574 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.199538 2578 generic.go:356] "Generic (PLEG): container finished" podID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerID="20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101" exitCode=0 Jul 24 13:06:26.199750 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.199591 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" Jul 24 13:06:26.199750 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.199599 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerDied","Data":"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101"} Jul 24 13:06:26.199750 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.199644 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf" event={"ID":"1178b5fa-2a16-4c34-a222-65d97019e2a0","Type":"ContainerDied","Data":"1bb71b960b195d1ce558746959c2096c258f7da89495e0ec24b7bd1539a756ed"} Jul 24 13:06:26.199750 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.199665 2578 scope.go:117] "RemoveContainer" containerID="20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101" Jul 24 13:06:26.209671 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.209649 2578 scope.go:117] "RemoveContainer" containerID="861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5" Jul 24 13:06:26.217038 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.217009 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:06:26.219549 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.219525 2578 scope.go:117] "RemoveContainer" containerID="20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101" Jul 24 13:06:26.219887 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:06:26.219866 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101\": container with ID starting with 20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101 not found: ID does not exist" containerID="20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101" Jul 24 13:06:26.219957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.219898 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101"} err="failed to get container status \"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101\": rpc error: code = NotFound desc = could not find container \"20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101\": container with ID starting with 20b986c86fa257c0c67af8691a373e188e22d463de9c4e4b7d224ba6416cc101 not found: ID does not exist" Jul 24 13:06:26.219957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.219923 2578 scope.go:117] "RemoveContainer" containerID="861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5" Jul 24 13:06:26.220296 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:06:26.220272 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5\": container with ID starting with 861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5 not found: ID does not exist" containerID="861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5" Jul 24 13:06:26.220402 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.220304 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5"} err="failed to get container status \"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5\": rpc error: code = NotFound desc = could not find container \"861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5\": container with ID starting with 861dab23e9eda72b0c3c93abb3d437b89aed30d978069fac557bf0a8b254a7b5 not found: ID does not exist" Jul 24 13:06:26.220804 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:26.220785 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-30955e-openshift-default-df7cb544b-5bdqf"] Jul 24 13:06:28.148208 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:06:28.148146 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" path="/var/lib/kubelet/pods/1178b5fa-2a16-4c34-a222-65d97019e2a0/volumes" Jul 24 13:07:38.594823 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:38.594777 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:07:38.595358 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:38.595089 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" containerID="cri-o://622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b" gracePeriod=30 Jul 24 13:07:43.859609 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.859528 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:07:43.904476 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904441 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904476 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904486 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6jctw\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904533 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904580 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904609 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904639 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904702 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.904738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904734 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.905041 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904784 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket\") pod \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\" (UID: \"74db8f88-b347-43d4-af48-4e8fe1ad7a5e\") " Jul 24 13:07:43.905041 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.904805 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:07:43.905153 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905030 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:07:43.905153 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905142 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:43.905307 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905159 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:43.905307 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905258 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:07:43.905424 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905386 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:07:43.906039 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.905767 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data" (OuterVolumeSpecName: "istio-data") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:07:43.907435 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.907407 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:07:43.907577 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.907561 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:07:43.907892 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.907861 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token" (OuterVolumeSpecName: "istio-token") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:07:43.908024 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:43.907891 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw" (OuterVolumeSpecName: "kube-api-access-6jctw") pod "74db8f88-b347-43d4-af48-4e8fe1ad7a5e" (UID: "74db8f88-b347-43d4-af48-4e8fe1ad7a5e"). InnerVolumeSpecName "kube-api-access-6jctw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:07:44.005957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005915 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.005957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005945 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.005957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005956 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.005957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005964 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.006265 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005972 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.006265 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005982 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.006265 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.005989 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6jctw\" (UniqueName: \"kubernetes.io/projected/74db8f88-b347-43d4-af48-4e8fe1ad7a5e-kube-api-access-6jctw\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:07:44.660478 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.660440 2578 generic.go:356] "Generic (PLEG): container finished" podID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerID="622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b" exitCode=0 Jul 24 13:07:44.660662 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.660491 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" Jul 24 13:07:44.660662 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.660534 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerDied","Data":"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b"} Jul 24 13:07:44.660662 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.660592 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d" event={"ID":"74db8f88-b347-43d4-af48-4e8fe1ad7a5e","Type":"ContainerDied","Data":"55c110654d24ba46919470ddd139461079dac8b5edbd864e75347d288240b584"} Jul 24 13:07:44.660662 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.660623 2578 scope.go:117] "RemoveContainer" containerID="622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b" Jul 24 13:07:44.670566 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.670544 2578 scope.go:117] "RemoveContainer" containerID="bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441" Jul 24 13:07:44.678081 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.678037 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:07:44.681491 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.681463 2578 scope.go:117] "RemoveContainer" containerID="622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b" Jul 24 13:07:44.681866 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:07:44.681843 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b\": container with ID starting with 622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b not found: ID does not exist" containerID="622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b" Jul 24 13:07:44.681946 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.681879 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b"} err="failed to get container status \"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b\": rpc error: code = NotFound desc = could not find container \"622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b\": container with ID starting with 622c0451de11ae19385172db4026ced7a194012fd10227f9708049727763594b not found: ID does not exist" Jul 24 13:07:44.681946 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.681905 2578 scope.go:117] "RemoveContainer" containerID="bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441" Jul 24 13:07:44.682167 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:07:44.682149 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441\": container with ID starting with bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441 not found: ID does not exist" containerID="bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441" Jul 24 13:07:44.682236 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.682175 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441"} err="failed to get container status \"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441\": rpc error: code = NotFound desc = could not find container \"bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441\": container with ID starting with bc0a59b474702a61a7efa098d8de53cd0ba8e96c7ddc0539c91b4f65e2e75441 not found: ID does not exist" Jul 24 13:07:44.682834 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:44.682816 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-967d3e-openshift-default-75b8f99bb-wz69d"] Jul 24 13:07:46.148764 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:07:46.148725 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" path="/var/lib/kubelet/pods/74db8f88-b347-43d4-af48-4e8fe1ad7a5e/volumes" Jul 24 13:08:00.604056 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:00.604013 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:08:00.604565 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:00.604304 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" containerID="cri-o://61ceae5e40f512748fb9d994957231ea3be982b056e45dea1dc6f0338a3b512d" gracePeriod=30 Jul 24 13:08:03.410804 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:03.410760 2578 patch_prober.go:28] interesting pod/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body= Jul 24 13:08:03.411227 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:03.410842 2578 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 24 13:08:05.786049 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:05.786016 2578 generic.go:356] "Generic (PLEG): container finished" podID="4c83c362-8c46-4633-bf95-d055b4074d52" containerID="61ceae5e40f512748fb9d994957231ea3be982b056e45dea1dc6f0338a3b512d" exitCode=0 Jul 24 13:08:05.786509 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:05.786092 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerDied","Data":"61ceae5e40f512748fb9d994957231ea3be982b056e45dea1dc6f0338a3b512d"} Jul 24 13:08:05.786509 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:05.786221 2578 scope.go:117] "RemoveContainer" containerID="3b3e4b5c5ef91818d17c2b6ecd6e583f04e6cac7892e10ef8fef8c4cf71a99ff" Jul 24 13:08:05.869255 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:05.869156 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:08:06.024542 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024506 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.024745 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024573 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.024745 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024610 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.024745 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024638 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fpm2j\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.024745 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024666 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.024745 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024692 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.025002 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024808 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.025002 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024863 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:06.025002 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024882 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.025002 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.024956 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert\") pod \"4c83c362-8c46-4633-bf95-d055b4074d52\" (UID: \"4c83c362-8c46-4633-bf95-d055b4074d52\") " Jul 24 13:08:06.025242 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025026 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:06.025334 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025313 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.025393 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025340 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.025456 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025432 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:06.025786 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025766 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:08:06.025826 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.025795 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data" (OuterVolumeSpecName: "istio-data") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:06.027167 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.027141 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:06.027295 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.027235 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token" (OuterVolumeSpecName: "istio-token") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:06.027508 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.027476 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j" (OuterVolumeSpecName: "kube-api-access-fpm2j") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "kube-api-access-fpm2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:06.027508 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.027485 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "4c83c362-8c46-4633-bf95-d055b4074d52" (UID: "4c83c362-8c46-4633-bf95-d055b4074d52"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126570 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/4c83c362-8c46-4633-bf95-d055b4074d52-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126602 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126610 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/4c83c362-8c46-4633-bf95-d055b4074d52-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126620 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fpm2j\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-kube-api-access-fpm2j\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126631 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/4c83c362-8c46-4633-bf95-d055b4074d52-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126639 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.126667 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.126646 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/4c83c362-8c46-4633-bf95-d055b4074d52-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:06.792717 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.792683 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" Jul 24 13:08:06.793227 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.792680 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578" event={"ID":"4c83c362-8c46-4633-bf95-d055b4074d52","Type":"ContainerDied","Data":"5c7c287cd172849c013fed4442637553d8f65e52812924dd1f938657a3b1df96"} Jul 24 13:08:06.793227 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.792820 2578 scope.go:117] "RemoveContainer" containerID="61ceae5e40f512748fb9d994957231ea3be982b056e45dea1dc6f0338a3b512d" Jul 24 13:08:06.809009 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.808968 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:08:06.814420 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:06.814381 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-47a7ae-openshift-default-75587fc998-gv578"] Jul 24 13:08:08.148871 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:08.148836 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" path="/var/lib/kubelet/pods/4c83c362-8c46-4633-bf95-d055b4074d52/volumes" Jul 24 13:08:17.450983 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:17.450947 2578 scope.go:117] "RemoveContainer" containerID="dccbc3e91d29c03459bc54827a9b2d38dcd95d7e30104a59f896ada21dfa7a2b" Jul 24 13:08:22.992687 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:22.992644 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 13:08:22.993124 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:22.992919 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" containerID="cri-o://97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9" gracePeriod=30 Jul 24 13:08:28.272480 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.272450 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 13:08:28.323756 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.323719 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4fqtz\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.323933 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.323770 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.323933 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.323803 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.323928 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324021 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.323988 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324131 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324069 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324131 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324095 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324131 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324119 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324336 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324145 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket\") pod \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\" (UID: \"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70\") " Jul 24 13:08:28.324336 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324206 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:28.324453 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324433 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.324453 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324430 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:28.324688 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324665 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:28.324887 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324788 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:08:28.324964 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.324900 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data" (OuterVolumeSpecName: "istio-data") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:28.326333 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.326309 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz" (OuterVolumeSpecName: "kube-api-access-4fqtz") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "kube-api-access-4fqtz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:28.326510 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.326483 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:08:28.326811 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.326788 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token" (OuterVolumeSpecName: "istio-token") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:28.326946 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.326928 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" (UID: "3619f5f2-545c-4ed0-84f9-b8c4d63bdf70"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:28.425611 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425571 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425611 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425602 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425611 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425611 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425611 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425618 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425883 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425627 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4fqtz\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-kube-api-access-4fqtz\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425883 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425636 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425883 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425644 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.425883 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.425651 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:28.920519 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.920480 2578 generic.go:356] "Generic (PLEG): container finished" podID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerID="97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9" exitCode=0 Jul 24 13:08:28.920519 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.920524 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" Jul 24 13:08:28.920773 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.920562 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerDied","Data":"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9"} Jul 24 13:08:28.920773 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.920649 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw" event={"ID":"3619f5f2-545c-4ed0-84f9-b8c4d63bdf70","Type":"ContainerDied","Data":"9f03c78e2932e07d715eef2a647a1561f15e1b28b74fd5bf70909b0891084158"} Jul 24 13:08:28.920773 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.920672 2578 scope.go:117] "RemoveContainer" containerID="97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9" Jul 24 13:08:28.930973 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.930950 2578 scope.go:117] "RemoveContainer" containerID="7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec" Jul 24 13:08:28.940612 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.940586 2578 scope.go:117] "RemoveContainer" containerID="97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9" Jul 24 13:08:28.940956 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:08:28.940933 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9\": container with ID starting with 97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9 not found: ID does not exist" containerID="97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9" Jul 24 13:08:28.941033 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.940965 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9"} err="failed to get container status \"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9\": rpc error: code = NotFound desc = could not find container \"97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9\": container with ID starting with 97d9e163cdc04bc348b07b442ae6b78e7cb0a86ce4cbf3ed2156be29f6f2b4a9 not found: ID does not exist" Jul 24 13:08:28.941033 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.940986 2578 scope.go:117] "RemoveContainer" containerID="7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec" Jul 24 13:08:28.941260 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:08:28.941236 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec\": container with ID starting with 7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec not found: ID does not exist" containerID="7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec" Jul 24 13:08:28.941345 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.941270 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec"} err="failed to get container status \"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec\": rpc error: code = NotFound desc = could not find container \"7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec\": container with ID starting with 7293fef0207c4d0519e9c32b23965701094bffce35a6efe13c5b3815384df3ec not found: ID does not exist" Jul 24 13:08:28.941682 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.941666 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 13:08:28.945690 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:28.945649 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-d15f64-openshift-default-565ccfdbd4-kb5rw"] Jul 24 13:08:30.148821 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:30.148782 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" path="/var/lib/kubelet/pods/3619f5f2-545c-4ed0-84f9-b8c4d63bdf70/volumes" Jul 24 13:08:44.952737 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:44.952696 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 13:08:44.953397 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:44.952963 2578 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" containerID="cri-o://db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41" gracePeriod=30 Jul 24 13:08:45.721389 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:45.721350 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7996f87bb5-kgxxn_d2f3d971-6797-4dff-a3aa-6c362be25d13/authorino/0.log" Jul 24 13:08:50.214134 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.214107 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 13:08:50.322293 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322254 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322293 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322298 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322556 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322319 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322556 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322352 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322556 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322393 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322556 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322422 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q26t\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322556 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322523 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322812 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322606 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322812 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322608 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:50.322812 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322645 2578 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy\") pod \"2190551a-80a6-4814-b040-e5a07aab6cf0\" (UID: \"2190551a-80a6-4814-b040-e5a07aab6cf0\") " Jul 24 13:08:50.322961 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322810 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:50.322961 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322850 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:50.323065 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322971 2578 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-credential-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.323065 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322993 2578 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-certs\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.323065 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.323006 2578 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-workload-socket\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.323065 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.322971 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 24 13:08:50.323444 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.323422 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data" (OuterVolumeSpecName: "istio-data") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:50.325121 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.325092 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 24 13:08:50.325267 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.325116 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t" (OuterVolumeSpecName: "kube-api-access-4q26t") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "kube-api-access-4q26t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:50.325490 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.325466 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token" (OuterVolumeSpecName: "istio-token") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 24 13:08:50.325553 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.325499 2578 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2190551a-80a6-4814-b040-e5a07aab6cf0" (UID: "2190551a-80a6-4814-b040-e5a07aab6cf0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Jul 24 13:08:50.424299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424247 2578 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2190551a-80a6-4814-b040-e5a07aab6cf0-istiod-ca-cert\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.424299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424286 2578 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-envoy\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.424299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424295 2578 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-data\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.424299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424302 2578 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-token\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.424299 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424312 2578 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q26t\" (UniqueName: \"kubernetes.io/projected/2190551a-80a6-4814-b040-e5a07aab6cf0-kube-api-access-4q26t\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:50.424620 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:50.424321 2578 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2190551a-80a6-4814-b040-e5a07aab6cf0-istio-podinfo\") on node \"ip-10-0-132-126.ec2.internal\" DevicePath \"\"" Jul 24 13:08:51.048535 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.048494 2578 generic.go:356] "Generic (PLEG): container finished" podID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerID="db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41" exitCode=0 Jul 24 13:08:51.048725 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.048555 2578 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" Jul 24 13:08:51.048725 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.048577 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerDied","Data":"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41"} Jul 24 13:08:51.048725 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.048641 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5" event={"ID":"2190551a-80a6-4814-b040-e5a07aab6cf0","Type":"ContainerDied","Data":"73b90fe0f953c0e004e27aed5656dd4581efdd66a20c0a5db52cea8b02307dd9"} Jul 24 13:08:51.048725 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.048697 2578 scope.go:117] "RemoveContainer" containerID="db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41" Jul 24 13:08:51.058676 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.058653 2578 scope.go:117] "RemoveContainer" containerID="a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8" Jul 24 13:08:51.074982 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.074955 2578 scope.go:117] "RemoveContainer" containerID="db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41" Jul 24 13:08:51.075355 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:08:51.075331 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41\": container with ID starting with db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41 not found: ID does not exist" containerID="db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41" Jul 24 13:08:51.075440 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.075367 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41"} err="failed to get container status \"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41\": rpc error: code = NotFound desc = could not find container \"db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41\": container with ID starting with db5d095ab9e81c0ef0d2316e98c88cd7392359b99e30bdee87d43aef3a4b8b41 not found: ID does not exist" Jul 24 13:08:51.075440 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.075388 2578 scope.go:117] "RemoveContainer" containerID="a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8" Jul 24 13:08:51.075624 ip-10-0-132-126 kubenswrapper[2578]: E0724 13:08:51.075609 2578 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8\": container with ID starting with a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8 not found: ID does not exist" containerID="a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8" Jul 24 13:08:51.075672 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.075627 2578 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8"} err="failed to get container status \"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8\": rpc error: code = NotFound desc = could not find container \"a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8\": container with ID starting with a43e43bd6ae0508d86c82f6b555b6f4603406472dc96660eaabf0d7dbcf8f2e8 not found: ID does not exist" Jul 24 13:08:51.076994 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.076966 2578 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 13:08:51.082652 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:51.082619 2578 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-c6697a-openshift-default-66b45f46cb-6v6j5"] Jul 24 13:08:52.025678 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:52.025646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c4d787c-brg96_053ad4b0-bb32-41aa-86ec-375a95743f7d/copy-manifests/0.log" Jul 24 13:08:52.032384 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:52.032347 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c4d787c-brg96_053ad4b0-bb32-41aa-86ec-375a95743f7d/manager/0.log" Jul 24 13:08:52.148422 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:52.148387 2578 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" path="/var/lib/kubelet/pods/2190551a-80a6-4814-b040-e5a07aab6cf0/volumes" Jul 24 13:08:52.826461 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:52.826429 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748270-6qvqd_1eacb9b8-d9b3-4e3e-8e8e-4c5aafd2840b/cleanup/0.log" Jul 24 13:08:52.938267 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:52.938238 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-key-cleanup-29748285-mjlvh_e5e5ab11-5307-4580-b385-4b5a6256c66d/cleanup/0.log" Jul 24 13:08:54.002709 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.002682 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/util/0.log" Jul 24 13:08:54.008957 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.008935 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/pull/0.log" Jul 24 13:08:54.014601 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.014581 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/extract/0.log" Jul 24 13:08:54.129878 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.129849 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/util/0.log" Jul 24 13:08:54.135160 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.135140 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/pull/0.log" Jul 24 13:08:54.140562 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.140537 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/extract/0.log" Jul 24 13:08:54.256359 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.256281 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/util/0.log" Jul 24 13:08:54.264066 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.264038 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/pull/0.log" Jul 24 13:08:54.271385 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.271361 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/extract/0.log" Jul 24 13:08:54.387408 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.387379 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/util/0.log" Jul 24 13:08:54.393822 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.393797 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/pull/0.log" Jul 24 13:08:54.414893 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.414868 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/extract/0.log" Jul 24 13:08:54.529711 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.529683 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7996f87bb5-kgxxn_d2f3d971-6797-4dff-a3aa-6c362be25d13/authorino/0.log" Jul 24 13:08:54.651584 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.651557 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h7b9d_5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036/manager/0.log" Jul 24 13:08:54.770624 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.770593 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rpts8_9cfcb201-4fd9-4c8f-a5e9-929f948eb45c/manager/0.log" Jul 24 13:08:54.888840 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:54.888764 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zqzf7_bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5/kuadrant-console-plugin/0.log" Jul 24 13:08:55.019432 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:55.019400 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vn6bp_eec1bc19-9682-4c58-9e2c-6e09a11114b8/registry-server/0.log" Jul 24 13:08:55.429495 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:55.429467 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dhxm8_92fe4503-a7d9-47f4-8ade-cce49d223dfd/manager/0.log" Jul 24 13:08:55.817675 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:55.817646 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf_19a0e2b6-d80e-45c7-8527-b1e010a76eb7/istio-proxy/0.log" Jul 24 13:08:56.346433 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:56.346399 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/1.log" Jul 24 13:08:57.113660 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:57.113634 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-89q7h_84afbe14-8890-4a80-ae3f-40340f25b455/main/0.log" Jul 24 13:08:57.378926 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:08:57.378846 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-pgbmg_1e600707-b82c-40cf-96eb-0096fd0a2e4b/main/0.log" Jul 24 13:09:06.333376 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:06.333343 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w7wlm_bf78f542-b932-49f4-9476-ed5cccaaaccd/global-pull-secret-syncer/0.log" Jul 24 13:09:06.422759 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:06.422724 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-ctvkd_1df7740b-666b-4ecb-87d4-d20b855117f7/konnectivity-agent/0.log" Jul 24 13:09:06.515300 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:06.515271 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-132-126.ec2.internal_dfcbafe564693f13913bc13ffce62314/haproxy/0.log" Jul 24 13:09:10.813397 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.813363 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/extract/0.log" Jul 24 13:09:10.842827 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.842791 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/util/0.log" Jul 24 13:09:10.865035 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.865010 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759vpgxz_192d104b-c5b9-46ba-8b7f-1e20cfa18300/pull/0.log" Jul 24 13:09:10.890995 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.890966 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/extract/0.log" Jul 24 13:09:10.911940 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.911914 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/util/0.log" Jul 24 13:09:10.934501 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.934473 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0b47rn_98ae81f5-4281-4ba0-b422-96b5540f5744/pull/0.log" Jul 24 13:09:10.959695 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.959671 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/extract/0.log" Jul 24 13:09:10.982935 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:10.982900 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/util/0.log" Jul 24 13:09:11.009050 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.009017 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73n7ccc_201d23be-aaf9-4abe-a911-b4ec3e37ed0d/pull/0.log" Jul 24 13:09:11.064717 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.064628 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/extract/0.log" Jul 24 13:09:11.095816 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.095782 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/util/0.log" Jul 24 13:09:11.119940 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.119912 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef14sp4m_7c2a3a95-b9a8-476e-b0dc-00b91c8c1d47/pull/0.log" Jul 24 13:09:11.490247 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.490128 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7996f87bb5-kgxxn_d2f3d971-6797-4dff-a3aa-6c362be25d13/authorino/0.log" Jul 24 13:09:11.519850 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.519813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-h7b9d_5f08ffa8-d8d2-4fee-bd0c-0fc4fa430036/manager/0.log" Jul 24 13:09:11.544604 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.544576 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-rpts8_9cfcb201-4fd9-4c8f-a5e9-929f948eb45c/manager/0.log" Jul 24 13:09:11.580769 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.580738 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-zqzf7_bf7ec99f-3b8c-4fa5-a7ba-0d1305dc89b5/kuadrant-console-plugin/0.log" Jul 24 13:09:11.631073 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.631036 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-vn6bp_eec1bc19-9682-4c58-9e2c-6e09a11114b8/registry-server/0.log" Jul 24 13:09:11.917753 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:11.917715 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-dhxm8_92fe4503-a7d9-47f4-8ade-cce49d223dfd/manager/0.log" Jul 24 13:09:13.800186 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:13.800162 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxbzv_00a7b9f2-bef2-4620-b1e9-e6baf4caea22/node-exporter/0.log" Jul 24 13:09:13.827738 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:13.827705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxbzv_00a7b9f2-bef2-4620-b1e9-e6baf4caea22/kube-rbac-proxy/0.log" Jul 24 13:09:13.863139 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:13.863112 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lxbzv_00a7b9f2-bef2-4620-b1e9-e6baf4caea22/init-textfile/0.log" Jul 24 13:09:14.442302 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:14.442265 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-7f8d78f54b-z2z9g_b5c6dd55-ca59-443d-9467-d771aecc7c5d/prometheus-operator-admission-webhook/0.log" Jul 24 13:09:16.723247 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723184 2578 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn"] Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723562 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723576 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723583 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723588 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723598 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723607 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723613 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723618 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723631 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723636 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723642 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723646 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723654 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723659 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723670 2578 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.723712 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723677 2578 state_mem.go:107] "Deleted CPUSet assignment" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723733 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723742 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="3619f5f2-545c-4ed0-84f9-b8c4d63bdf70" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723748 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723757 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="2190551a-80a6-4814-b040-e5a07aab6cf0" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723767 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="1178b5fa-2a16-4c34-a222-65d97019e2a0" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723775 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723780 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="74db8f88-b347-43d4-af48-4e8fe1ad7a5e" containerName="istio-proxy" Jul 24 13:09:16.724214 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.723785 2578 memory_manager.go:368] "RemoveStaleState removing state" podUID="4c83c362-8c46-4633-bf95-d055b4074d52" containerName="istio-proxy" Jul 24 13:09:16.726854 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.726835 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.729319 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.729297 2578 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bz6zh\"/\"default-dockercfg-jsx6c\"" Jul 24 13:09:16.729608 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.729586 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bz6zh\"/\"openshift-service-ca.crt\"" Jul 24 13:09:16.729762 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.729745 2578 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bz6zh\"/\"kube-root-ca.crt\"" Jul 24 13:09:16.737741 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.737710 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn"] Jul 24 13:09:16.759606 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.759567 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qlgz5\" (UniqueName: \"kubernetes.io/projected/008f8185-d876-4115-8a2c-52c8973e1f06-kube-api-access-qlgz5\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.759838 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.759623 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-lib-modules\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.759838 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.759669 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-sys\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.759838 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.759702 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-podres\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.759838 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.759732 2578 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-proc\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861220 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861161 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-proc\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861401 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861253 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qlgz5\" (UniqueName: \"kubernetes.io/projected/008f8185-d876-4115-8a2c-52c8973e1f06-kube-api-access-qlgz5\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861401 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861304 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-proc\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861401 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861311 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-lib-modules\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861401 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861385 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-sys\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861417 2578 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-podres\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861452 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-lib-modules\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861500 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-sys\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.861550 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.861511 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/008f8185-d876-4115-8a2c-52c8973e1f06-podres\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:16.869741 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:16.869712 2578 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qlgz5\" (UniqueName: \"kubernetes.io/projected/008f8185-d876-4115-8a2c-52c8973e1f06-kube-api-access-qlgz5\") pod \"perf-node-gather-daemonset-xwnjn\" (UID: \"008f8185-d876-4115-8a2c-52c8973e1f06\") " pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:17.038173 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:17.038137 2578 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:17.182488 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:17.182443 2578 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn"] Jul 24 13:09:17.185865 ip-10-0-132-126 kubenswrapper[2578]: W0724 13:09:17.185832 2578 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod008f8185_d876_4115_8a2c_52c8973e1f06.slice/crio-a6a6668ba9d4f50f043cd3bb466d032f836fbf92617714308b32fb9f0f57ccba WatchSource:0}: Error finding container a6a6668ba9d4f50f043cd3bb466d032f836fbf92617714308b32fb9f0f57ccba: Status 404 returned error can't find the container with id a6a6668ba9d4f50f043cd3bb466d032f836fbf92617714308b32fb9f0f57ccba Jul 24 13:09:17.192647 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:17.192609 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" event={"ID":"008f8185-d876-4115-8a2c-52c8973e1f06","Type":"ContainerStarted","Data":"a6a6668ba9d4f50f043cd3bb466d032f836fbf92617714308b32fb9f0f57ccba"} Jul 24 13:09:17.252739 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:17.252705 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7c9b9647f6-4vgz8_8fa48eeb-c75e-4c4e-9819-cbe862f1588f/console/0.log" Jul 24 13:09:17.294005 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:17.293914 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-7c97dc4555-t5csq_f013bb68-2da6-44c1-a26c-c8f097390f8f/download-server/0.log" Jul 24 13:09:18.199609 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:18.199568 2578 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" event={"ID":"008f8185-d876-4115-8a2c-52c8973e1f06","Type":"ContainerStarted","Data":"162db3f28c8677e7a5cd686cc27374d18224128f5d3c1d9b75c9e849b4ac9121"} Jul 24 13:09:18.200007 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:18.199640 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:18.217739 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:18.217686 2578 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" podStartSLOduration=2.217666497 podStartE2EDuration="2.217666497s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-24 13:09:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-24 13:09:18.215628228 +0000 UTC m=+3014.501173930" watchObservedRunningTime="2026-07-24 13:09:18.217666497 +0000 UTC m=+3014.503212229" Jul 24 13:09:18.994122 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:18.994082 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2g8z9_7ac784d3-ac39-4697-8db8-90e5b096374f/dns/0.log" Jul 24 13:09:19.017777 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:19.017744 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-2g8z9_7ac784d3-ac39-4697-8db8-90e5b096374f/kube-rbac-proxy/0.log" Jul 24 13:09:19.146024 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:19.145995 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4blg2_598c1f25-29f8-4ce5-acf0-362bcc9c4854/dns-node-resolver/0.log" Jul 24 13:09:19.731247 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:19.731214 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-7c9d66689c-9wthm_de704828-fa9e-41c7-a646-cc1d0db04e31/registry/0.log" Jul 24 13:09:19.755379 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:19.755349 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-hkz6p_e1086883-e355-4c97-9849-9efa8b112d13/node-ca/0.log" Jul 24 13:09:20.735411 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:20.735381 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf_19a0e2b6-d80e-45c7-8527-b1e010a76eb7/istio-proxy/0.log" Jul 24 13:09:21.097126 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:21.097089 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/0.log" Jul 24 13:09:21.123854 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:21.123813 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/1.log" Jul 24 13:09:22.047580 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:22.047550 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-jslk5_cecbdba0-1239-44c4-ba86-cb670f0e7f6d/serve-healthcheck-canary/0.log" Jul 24 13:09:22.739187 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:22.739154 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvrkq_11bf7252-0597-488c-a6f0-4711e2e20ae0/kube-rbac-proxy/0.log" Jul 24 13:09:22.758305 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:22.758267 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvrkq_11bf7252-0597-488c-a6f0-4711e2e20ae0/exporter/0.log" Jul 24 13:09:22.778417 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:22.778371 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-zvrkq_11bf7252-0597-488c-a6f0-4711e2e20ae0/extractor/0.log" Jul 24 13:09:24.215033 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:24.215003 2578 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bz6zh/perf-node-gather-daemonset-xwnjn" Jul 24 13:09:25.633470 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:25.633435 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c4d787c-brg96_053ad4b0-bb32-41aa-86ec-375a95743f7d/manager/0.log" Jul 24 13:09:25.659327 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:25.659301 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-565c4d787c-brg96_053ad4b0-bb32-41aa-86ec-375a95743f7d/copy-manifests/0.log" Jul 24 13:09:27.227777 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:27.227694 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-d4b69f9c5-52nlj_a5d917e4-7933-4b9b-b63a-5219edd81908/manager/0.log" Jul 24 13:09:27.382558 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:27.382523 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf_19a0e2b6-d80e-45c7-8527-b1e010a76eb7/istio-proxy/0.log" Jul 24 13:09:27.727442 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:27.727406 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/0.log" Jul 24 13:09:27.755901 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:27.755862 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/1.log" Jul 24 13:09:29.254289 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:29.254258 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-z6wpf_19a0e2b6-d80e-45c7-8527-b1e010a76eb7/istio-proxy/0.log" Jul 24 13:09:29.588346 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:29.588310 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/0.log" Jul 24 13:09:29.620276 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:29.620235 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-hpjpm_59ddf789-0df8-4c83-b607-33d3954167e4/istio-proxy/1.log" Jul 24 13:09:32.436624 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:32.436590 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-j84gd_2647aee1-a03d-448d-9597-401ac14cf241/migrator/0.log" Jul 24 13:09:32.457054 ip-10-0-132-126 kubenswrapper[2578]: I0724 13:09:32.457021 2578 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-6865c9967-j84gd_2647aee1-a03d-448d-9597-401ac14cf241/graceful-termination/0.log"